HouseRalan HouseRalan

New Mem Limit, hardcode limits?

New Mem Limit, hardcode limits?

What've got to look forward to in Rebellion?

I don't know the technical side of what's planned for Rebellion.  I know there won't be a 64-bit executable, though I don't know why really or what the hurdles are.

What will be the memory limit?  SOASE only uses 2gb of memory, right?  After it reaches the 2GB limit, POOF; SOASE dumps.  Will Rebellion have a mem crash limit of 3gb, 3.5, 4, 50?  Where will Rebellion hit the wall and explode into flames? 

What are projected sound, mesh, etc... hardcode limits or will it be the memory limit that controls all things?  In particular, I'm asking as a modder of 7 Deadly Sins.  We currently have 11 playable races + NPC Pirates, 2,199 entities in GameInfo, 452 mesh files in mesh, 689 in particle, 1,377 in Sound, 1,393 in textures.  We have set a good number of things as 'load on demand'.  A 4 player game on medium to large map with planets medium, ships all medium, structures all on medium easily reaches 1.7 GB of memory.  More detail and that 2Gb limit hits fast; more players -same issue.  So, just given the file quantities above, 4 player settings on a medium/large map, and assuming we easily get 7DS into Rebellion (easier than reaching Beta in Entrenchment), will we still have the dreaded 2GB memory limit issue?  Will we finally get to have 8 or 10 races playing your huge, random, multi-star map with all settings on medium?

Gotta know what Rebellion holds for us as well as other mods that plan 6+ races with their own meshes, sounds, textures, etc... The LARGE mods.  We are considering what it'll take to go fully modular, where we can stack in exactly which races the player wants to play so they can only load in the specific races' meshes, sounds, textures (or more, we have more planned), but that means figuring out how to deal with GalaxyScenarioDef, sounddata, and a variety of other files as well as entity.manifest.

So StarClad, what've we got to look forward to in Rebellion so we can start figuring out options now?

105,861 views 53 replies
Reply #26 Top

Sins 2 will be another thing all together. I would expect Sins 2 if it actually manifests itself to be a 64 bit multicore capable game. Packed with all of the bells and whistles that come with it.

Right now we are talking about Rebellion, and the Iron engine that is starting to become dated. Dated only because of the fact that it is 32 bit. TSOP pretty much proved that more can be crammed into the iron engine in its current state. With strategic optimizations of both the code, and the textures. IC took care of the code with the diplomacy 1.2 patch.

The thing you have to ask is just because you can use a 4096 resolution texture, does it mean that you "should" use it? Especially if a 2048 resolution texture yields the same, or similar results. Do you really need a 512 res texture for a particle that is the size of a pinpoint in game?

Another thing to look at is why there is late game cpu lag. Yes, CPU lag. The very same problem Supreme Commander had until recently. TSOP fixed some of the video lag problems, but the CPU lag is due to a game engine optimized for single core 32 bit cpu's. and way too many units on the map for our quad core cpu's that only use 1 core to play sins to handle. 1 core cannot handle thousands of units on a map along with calculating those thousands of units movement, weapons, buff's, debuff's, and ability's. Along with the particles, and effects involved.

The Iron engine handles meshes beautifully as in high poly counts with little hit on performance. That is what really impressed me with this engine.

If the existing engine can be worked over to use multiple cores i think we will have 2/3rds of the problems solved.

Reply #27 Top

The issue is that 64-bit CPU's are the standard now, I don't think even the budget CPU's in budget computers on the shelf now are even x86 CPU's. The standard for most games is to at least use two cores on a CPU, and some even more. RAM, CPU and GPU are all vital bottlenecks in game speed (to some degree HDD as well). If you have a current GPU but DDR2 RAM and a single core CPU, you are not going to get all you can from modern games.

 

If you have a computer from 5+ years ago and insist on gaming with it, prepare to be disappointed. Your old CPU doesn't have the latest instruction sets such as SSE3, your old RAM is crawlingly slow and your video card might as well run on coal. It's kind of like expecting a Xbox 360 game to run on an Xbox. I see no reason why modern games should be held back by "fossil" tech. It's the year 2011 for crying out loud, single cores are so last decade.

Reply #28 Top

Quoting Major, reply 26
If the existing engine can be worked over to use multiple cores i think we will have 2/3rds of the problems solved.

I recall, at some point, perhaps it was merely a suggestion/rumor, to make the engine work by rendering certain things on different cores. Ex. Ships on one core, with planets on another. Not sure if it's even possible with the engine, though it would make for an interesting workaround which may help.

Reply #29 Top

First, Ironclad designer had budget for model, texture and other thing so in the end, the game were running smootly. Mod dev should try to stay within theses budgets. 

 

Every game engine will look unoptimized if you overide the budget. Take the latest unreal engine, put full of 10 millions polygons models and you will kill the performance so bad that it could crash at launch !

 

Anyway, for the x64 vs 32bit, if we take a look to the steam survey (yeah competitor.. but still good information) ;)

 

http://store.steampowered.com/hwsurvey

In april, 40% of the system were using Win 7 x64, if you add vista x64, that is 51% of gaming system that run a 64 bit system. (i will not add winxp x64 since its outdated and do not support dx11 as well), Also, count that 3.5 of all machine are running OSX. 

 

Win XP 32 bit have only 21% of the share and it loose more and more of the share. Dual and Quad-core represent 90% of the share. big majority here for multithreading processor. most system have 3 GB or more of ram. 

 

So i think that for a game planned for 2012 that is not a reworked game (like rebellion will be) should be x64 and directx11 ready.

 

By the way, using the directX11 direct3d engine (that is fully multithreaded) do not imply to only support DirectX11 card. DirectX 10 and by a lower extend, directX9 card can run the directX 11 Direct3d engine. The game will just select the right shader model for the card. (but sure older card will not be able to run the game at max detail).

 

Battlefield 3 will be the first major game to be fully based on directX 11 and will have a x64 support also. but dev will still have to carefully manage polygons, shaders exec time and texture weight budget to make a smooth optimized game.  

 

(ps. sorry for my bad english, not my first language)

Reply #30 Top

Some necro post...

Quoting Major, reply 26
The Iron engine handles meshes beautifully as in high poly counts with little hit on performance. That is what really impressed me with this engine.

Very true... a recent test with my space station model... look at the video at http://www.love-from-russia.be/g4_swarm-6.25_million_poly.avi ... 125 mobile starbase, with each starbase being around 50k poly, with 3 rotating part ( small neutron star and the two ring ), over 200 weapons hardpoint, 4 set of texture map, etc... and no slowdow...

As for the opening post :


...What will be the memory limit?  SOASE only uses 2gb of memory, right?  After it reaches the 2GB limit, POOF; SOASE dumps.  Will Rebellion have a mem crash limit of 3gb, 3.5, 4, 50?  Where will Rebellion hit the wall and explode into flames? 
...
In particular, I'm asking as a modder of 7 Deadly Sins.

...

A 4 player game on medium to large map with planets medium, ships all medium, structures all on medium easily reaches 1.7 GB of memory.  More detail and that 2Gb limit hits fast; more players -same issue.

...

Will we still have the dreaded 2GB memory limit issue?  Will we finally get to have 8 or 10 races playing your huge, random, multi-star map with all settings on medium?

Samaz, the 2gb thing is not a iron engine limit but a OS limit...

Today, i have start a huge multistar map, with 10 races and pirates using 7DS 2.1 ... all setting on very high : this alone crash directly when using windows...

Now, look at the screenshot below :

Game is using a little over 3.5 gb ( 175% more that the so called "2gb sins limit" )... along with a little over 20 mb for wine ( windows compatibility layer )... it is a non modified sins.exe ( not LAA and limited to a max of 4gb )... the only difference is a other OS... Linux ( Kubuntu )... Stardock/Ironclad don't support Linux but sins of a solar empire work better on Linux that on Windows...

The point is that a lot of the sins limit is not related to the iron engine but to the layer below that people call OS...

NOTE : multiplayer work on linux ( need mono for a .NET compatible layer )... Impulse don't work ( unless you have enough linux knowledge for install/compile wine with "winetricks dotnet20 gdiplus" who solve the login problem )... so, for a Joe user, windows is needed for patch... once patched, copy the full folder from windows to Linux...

INFO : http://appdb.winehq.org/objectManager.php?sClass=version&iId=19966 ( trinity on Linux )

Reply #31 Top

I had the problem with SupCom:FA a while ago and managed to edit the adressed ram with CHEATENGINE. (which shouldnt be possible if it would be true that a 32bit application can't handle more than 2gb of ram). But now there are even tools to edit any program executable to enable 4gb of ram (only if you are playing on a 64bit system or an old 32bit system with 3gb cap limit changes in boot.ini of course).

Just to name TWO of the dozens of programs that can do this:

http://www.ntcore.com/4gb_patch.php

http://www.techpowerup.com/forums/showthread.php?t=112556

 

It's stupid to say it wouldn't be possible to use more memory on a 32bit application. -.- Stop lying to people! I'm playing large 5on5 matchs with my friends without problems and reach about 2-3gb memory in lategame which doesn't crash the game thanks to the tools above. (for 32bit systems don't forget to edit you boot.ini to allow 3gb of memory for one executable) I'm on win7 64bit right now and it works fine all of the time. :)

Reply #32 Top

everfades, ARE you one of the engine developers for the iron engine?

if NOT, then the LAA FLAG ON works ONLY on 64 bit windows AND STILL only ALLOWS ram use by the 32bit program to reach 4gb before crash  ON 64 bit windows, and with the laa flag off OR 32 bit windows, the program is STILL limited to 2gb.

if you ARE then WHY THE FUCK have you not supplied a 64 bit version of the iron engine???

the cheatengine would be using the LAA flag to do this.

and for your read edification, the MAIN limitation in sins is the LAG at mid-late game from all the thousands of strikecraft, trade ships, frigates,cap ships starbases, structures,weapon shots, particles, effects AND titans that NEED to be updated every fraction of a second, and only one of the causes of the minidumps IS too many ships/structures/strikecraft/starbases/particles/effects, the more common during mod development/use IS minor errors in the mod's files compared with that the engine expects and wants.

harpo

 

Reply #33 Top

Quoting harpo99999, reply 33
with the laa flag off OR 32 bit windows, the program is STILL limited to 2gb.

 

Yeah, cheatengine can enable the LAA flag, too. But once again: Wrong! Enable the LAA flag (with one of the programs from my last post) and do this if you have windows xp 32bit.

 

1. Right-click My Computer and select Properties. The System Properties dialog box will appear.

2. Click the Advanced tab.

3. In the Startup and Recovery area, click Settings. The Startup and Recovery dialog box will appear.

4. In the System startup area, click Edit. This will open the Windows boot.ini file in Notepad.

5. In the [Operating Systems] section, add the following switches to the end of the startup line that includes the /fastdetect switch: /3GB

6. Save the changes and close Notepad.

7. Click OK two times to close the open dialog boxes, and then restart the computer for the change to take effect.

 

Et voila - 3gb limit on a 32bit OS. :)  Enjoy! ;)

Reply #34 Top

Quoting harpo99999, reply 33
and with the laa flag off OR 32 bit windows, the program is STILL limited to 2gb

While by default it is a 2GB/2GB split, it is possible to configure 32-bit versions of windows to restrict the kernel to only 1GB of the available 4GB address space and make the remaining 3GB available to applications (though they must be LAA to take advantage of this).

Reply #35 Top

everfade, the other problem with LAA on 32 bit windows is the REDUCED stability of the operating system, ie it is MORE likely to crash, so go ahead adnd do it for your self, just do NOT make out like you are a programming god that knows all, UNLESS you ARE a programmer of over 5 years experience with all types of programs on a wide variaty of operating systems.

harpo

 

Reply #36 Top

Minimising the amount of resources required is often the best approach to these kinds of problems.

In addition to reducing memory requirements, such optimizations usually also improve performance - TSOP proved this to such an extent that many of its changes were incorporated in to Diplomacy itself.

It is these kinds of refinements that make Sins playable on older hardware, a consideration that you don't often see in today's gaming market.

However, from reading the MSDN article Thoumsin pointed to it would be interesting to see what would happen if the dev's enabled the /LARGEADDRESSAWARE linker flag when compiling Sins - I'm assuming Visual Studio is used.

This would give users of x86-64 systems, and those x86-32 systems with boot.ini changes, the capability of using more memory, without (hopefully) affecting the compatibility of unmodified 32-bit systems and multiplayer.

 

Reply #37 Top

the main effect is to delay the point of minidump from out of application data space, BUT will have no effect of the ram/cpu thrash and LAG that the tens to HUNDREDS of THOUSANDS of strikecraft/ships/mines cause.

harpo

 

Reply #38 Top

Quoting NewHorizons, reply 37
it would be interesting to see what would happen if the dev's enabled the /LARGEADDRESSAWARE linker flag when compiling Sins

It was tested and deemed to add too much instability.

Reply #39 Top

Who the hell got used because today there are 32 bit OS I W7 Ulti 64 bit version

and 16 GB of ram at the prices at the moment a must. O:)

Reply #40 Top

Quoting kryo, reply 39
It was tested and deemed to add too much instability.

Ahh well, it was worth a shot - seemed a solution that might have satisfied a few on the forum without too many adverse side effects.

As harpo99999 points out, you would need one hell of a fast (single) core to cope with the larger numbers of objects that can be stored in the increased address space.

Reply #41 Top

Wait, what? I was about to buy Rebellion, but i thought better check the Forums first to see what people say versus what is being advertised (as it didnt really tell me that much) and here i read there is no 64bit nor multi-core support (what i first assumed, but did not really know from the advertisement).

Anyway as much as i appreciate this game and the developers, for that price, im properly gonna wait for Sins2.

 

PS: Also interesting, that it runs better under Wine.

Only recently started with Linux myself and so far it has been great.

Never again windows except gaming.

Altho i hear valve is working hard at getting over there, too. (natively)

Reply #42 Top

Quoting Omicron2k5, reply 42
Wait, what? I was about to buy Rebellion, but i thought better check the Forums first to see what people say versus what is being advertised (as it didnt really tell me that much) and here i read there is no 64bit nor multi-core support (what i first assumed, but did not really know from the advertisement).

Anyway as much as i appreciate this game and the developers, for that price, im properly gonna wait for Sins2.

 

PS: Also interesting, that it runs better under Wine.

Only recently started with Linux myself and so far it has been great.

Never again windows except gaming.

Altho i hear valve is working hard at getting over there, too. (natively)

 

Well, it is your loss.

 

On a more serious note, the ammount of games that make use of multi core is quite limited, even nowadays. Run one of your games for a bit after starting task manager. Then watch the CPU usage off all cores. You are going to be dissapointed because many of them arent doing much.

 

Same for 64 Bit.... If your game does not use more than 2 GB of RAM at anytime... there is no point or advantage in running 64 bit.

Reply #43 Top

Quoting ARESIV, reply 43
there is no point or advantage in running 64 bit.

Not entirely true, compilers can generate more efficient code when using the x86-64 instruction set.

Reply #44 Top

Quoting ARESIV, reply 43
Same for 64 Bit.... If your game does not use more than 2 GB of RAM at anytime... there is no point or advantage in running 64 bit.

If your using 32bit XP you likely don't have a full 2GB available for your game even if you have 4GB of RAM installed.

 

Reply #45 Top

I wish there were something -a tutorial for win7 users to get their rigs running linux just for SOASE.  I personally want to got Linux/Wine/? for SOASE.

Reply #46 Top

Better still.  StarClad should start working on Trinity/Rebellion for Linux so we don't have to screw about to get better performance out of the game.

Reply #47 Top

 

Quoting SemazRalan, reply 47
Better still.  StarClad should start working on Trinity/Rebellion for Linux so we don't have to screw about to get better performance out of the game.

I certainly would be delighted.

PS: Installling Linux is not hard, if you already installed windows. Try and give Ubuntu a go, if you want to take a look. It is one of the most supported distributions out there.

PPS: There is also a live cd of it avaible or you can run it via a virtual machine. Although i do not know what limitations that has conerning performance. For gaming under wine it is probably best to let it run natively.

 

 

Reply #48 Top

I have a question, If 32bit will only allow 2GB per process, then why not add a second process? Would that be possible? or am I way off?

You start Sins and it has Sins A and Sins B processes, keeping the high demand items in Sins A, and storing lesser used things in Sins B.

Reply #50 Top

The overhead & complexity trying to do IPC (Inter Process Communication) for Sins would be horrific.

By comparison porting to x86-64 is fairly straight-forward, even multi-threading would probably be easier.