Yea, but it is no concern of programmer. The OS handles that, unless the programmer specifically wants to participate in the process. Despite your needs, there is no way a regular PC could move all that data to registers and do calculations with it at once. Even if there is data available worth of Xgb, you'll need only few kilobytes of that at once. That is why the virtual memory can work so independently.

It becomes the concern of the programmer as soon as someone creates a super stupid huge system on a machine with 4GB of RAM, and the game slows to a crawl as it pages in and out trying to do things that require more than 4GB of data in working memory at the same time, because that person is going to post a review on Metacritic complaining about how slow the game is.

Memory management got easier, but it hasn't gone away.

True, and that is why I said I'll simplify it. That is not a technical issue, it is a marketing and user issue. In reality the programmer probably really is forced to limit the parameters so that nobody tries to create the huge world which even a super computer couldn't run. But in the end, it is a choice between allowing things and not allowing, because RAM is not the real limit here.

You are both only part right. From a general system perspective the primary problem is how high the page fault rate goes. The more page faults per second the slower an app will run. It can even get so high that the program will appear hung. Providing more RAM will reduce the paging rate.

Yes, they got rid of the 2G barrier, but any program developer, game or otherwise, should know how sensitive their program is to available memory. In the case of GC3, I expect the devs to throttle the size of maps you can play based on how much page faulting they expect on each size. Tiny and Small maps may be playable on 4G of memory, but I very much doubt that you would be able to play a humongous map on 4G. Hopefully there will be testers in our community that will be able to test this out.

Your point? Lets assume you have 16gb ram in your computer and all that is available for the game. Do you seriously think your cpu could process all that all the time? Even the bus speed limits transfers to somewhere around 5-10gb/sec, so with that kind of processing you would get something like 2fps game. And that assumes you only transfer the data between cpu and ram modules without doing even a simple 1+1 calculations, which would reduce the frame rate even more. No, most of that data just sits there until it is needed and while doing that, it could just as well be in the hdd (or more preferably in ssd). Of course too much data there hurts, but we are talking about big rams now and not some little 2gb machine.

During a turn end when all the AIs are going, you need the entire game state in memory to do all the processing that comes with it. If you have to constantly page game state in and out to do AI turns, the turns will take dramatically longer because paging is horrifically slow.

It seems unlikely there's going to be 4GB of game state to work with for that, but we'll see.

First, as far as I have understood right, modern AI works all the time in the own thread(s) ( ) and not just when you press end turn. Second, I don't think the AI needs all the graphical data needed for planetary views for example, or enemy data it doesn't see, or objects waiting something before things happend etc etc. I'm no game programmer so I could be wrong about that, but I doubt because it is common sense. However, it doesn't change the fact that 16gb precious data doesn't compute well in such situation no matter where it is.

I'd say that with some of the galaxy sizes some people are asking for that the amount of game state could be quite a bit larger than 4GB. But we will have to see.

The stuff that happens at the end of a turn can't happen until the player pushes the end turn button, no matter how many threads you have. Also if turns are sequential the AI can't go at all until their turn comes around, no matter how many threads you have.

The AI can use lots of threads to figure out what it wants to do, but actually doing it is a sequential process.

Oh, but I think the keyword here is the 'figure'. I think the only unknown variable here is what player does do, and there is no reason why AI should wait for 'end turn' before reacting to what ever commands the player issues. As I see it, one could make the AI queue commands just like player can do regarding building, moving, researching, etc. Processing those commands in a sequential order on the 'end turn' is a whole different thing than the decision making. I don't think multiple AI's really have to play like players (sequential order), they just need to make it look like they do. The problem of course is, that there probably ain't that many AI controlled game elements in the game that just sit and wait without any need to check from time to time. However, I seriously doubt any game would have that kind of information worth over the 4gb. I mean how many things you need to describe for example a unit? Speed, shields, hull, that kind of things. Lets assume one would take worth of 4kb of data, which is really quite a lot. Now the 4gb would allow more than 1 million such units in the game, and to me it sounds rather big number (more than the cpu can handle).

Just to put something in perspective, I have windows 8.1 with the following up and running: Firefox, + Classic Shell, + Bitmeter (shows lan/net usage rates), + "7 Sidebar" with a clock, CPU Usage (also shows RAM usage, clock speed, and CPU temperatures), GPU Meter (shows GPU usage and GPU temperature), and Drives Meter, These alone use almost 3GB of RAM.

Steam adds another 200MB.

Bringing up GC2 TotA with an immense galaxy adds about another 700 MB.

So, if you only have 4GB of RAM, GC2 is already close to page faulting, which will start slowing down the game's performance, as we have discussed in other threads.

For comparison purposes, Windows 7 uses a bit less memory. I just checked my wife's PC, on which she has the sidebar with a clock, CPU Usage, GPU Usage, and Network Monitor, and Live Mail up and running using a bit more than 2.3GB RAM. FireFox adds about 300 MB to that.

I think we can see from this that the 4GB RAM minimum figure being talked about on these forums is an absolute bare minimum to run anything on a modern PC.

Trying to calculate how much GC3 is going to need just for ships is also very short sighted. Ship designs need very little memory compared to how much memory is needed just for a segment of the galaxy, with all of the stuff that must be put in each tile, such as stars, planets, ships, resources, anomalies, path lines for trade routes and asteroid mine delivery routes, tile borders, etc. etc. And that isn't all that GC3 is going to have in its address space. Oh, and don't forget all the executable code.

***********

EDIT---> I just found out that the game I am playing is a gigantic galaxy, not an immense one. I also tried loading a Tiny galaxy, just to see how much RAM would be allocated to it. It was about 150MB smaller than for a gigantic galaxy. Since the Tiny was a new game without any turns, and the Gigantic was after exploring the entire galaxy, seeing two major races die due to wars with other races, and having conquered half of the drengin's planets (about half a dozen planets), it is quite likely that any comparison between the memory usage is skewed a bit.

Ships in Galactic Civilizations 3 are going to have much higher polygon counts and axes of freedom; they will be vastly more complex and vastly more detailed if the founder's vault is to be believed. I wouldn't be so quick to dismiss ship designs as an insignificant memory burden just yet, especially on games with lots of custom ships. That being said, you're probably by-and-large correct. 

Yes, this is true, but I think you missed the keyword "just" in the first sentence. I wasn't saying that ship designs don't use memory, but that there are other things that use quite a bit more space. And ships aren't the only thing that are going to use more space in GC3 due to the improvements they have told us about so far.. So is everything else displayed in the galaxy that will have improvements. Let's just say that talking about how much RAM will be needed by GC3 solely on how many ships you have in your fleet is rather short sided.

Somehow I get the feeling that you don't want to see memory swapping at all. But the argument was, can it be allowed? I think it can be to some extent, and I have tried to explain why (because I think other issues with modern hardware ruin the game before that). I don't know where the exact barrier between playable and non-playable goes, but I can agree that 4GB of system ram is probably too little for a gaming computer. 

Btw. buy ssd if you already haven't. Reduces a lot of those loading times you mentioned 'can even get so high that the program will appear hung'.

No, I am not against memory swapping (also called page swapping since it is handled by the OS a page at a time, and is caused by page faulting). Page faulting and the resultant page (or memory) swapping is a natural function of virtual address space operations. A little bit can be considered good, as that means you haven't spent more money on RAM than you needed to.

But, when the page fault rate gets high the performance of the OS and all running apps can become unbearable, and even appear hung.

So, it becomes a balancing act. Which do you want? More paging or more memory?

As an added note, don't forget how much damage to performance AdWare can cause.

Of course having limits can have it's 'advantages'. I recall that for Fallen Enchantress (I think...) there was a point where there was debate as to whether the largest map type would be available due ot memory contraints for certain users. But after some hard optimization, that it was announced that said map size would be available.

In that situation the hard limit of the memory forced them to do some memory optimization. Without that 'wall' would they ahve bothered?

I'm certainly not saying that they're not going to optimize teh game at all. Just saying that not having limits is both liberating, and a crutch.

Disclaimer: As I cant' find teh original blog post, I could be misrepresenting, misremember, or just plain wrong about teh above situation as it relates to FE! If so I apologize. I still do stand by the 'general' idea that the 32bit wall may be both good/bad.

Yes, and nobody has tried to deny that fact. 

I have 16gb ram on my notebook and something like 12gb of that actually free. I'm not going to even try filling it with a regular game, swapping or no swapping. I don't care if the game allocates it all just for fun, but once it tries to make use of it all, then the things start to fall apart.

64-bit is okay and all, but 65-bit would be better.

And the beta party drinking party starts.....

All this talk about game playability [after all that's what it boils down to] is starting to worry me.  Even with my 4-core i7 processor, it sounds like that's where the choke point will be when the game is ready for release.

Maybe we will get some indication of how things will go later today with the Beta1 release, to see how things perform and to see whether the game will still C-T-D at around 270+ turns.

My rig has 16Gb system Ram and 4Gb of GPU Ram.  It is almost 2-years-old and I expect it to last another 3-years.  This means that my gaming experience with GC3 will be limited by my current hardware and will determine whether the game is a success for me or not.  Although I may upgrade my Video Card next year, and, if Stardock do follow up on their interest in multi-monitor set-ups, then I may invest in a second 27" screen to go with a more powerful Graphics card [whether this will affect the CPU performance is an unknown to me].

And I know I'm probably wrong, but with all the hype over 64bit O/S Windows, My understanding is : that it's not total 64bit, just 32bit with 64bit addressing to memory/RAM.  Btw I'm still using Win-7 O/S and don't expect to upgrade before at least Win-9 [whenever that may be].

Games today are already attempting to break the 32-bit limitations. Games that have heavy mod capabilities like Skyrim had to have LAA enabled so that more and more mods could be loaded. But in the end the 32-bit exe means that there is a hard limit as to how many mods and what you can do at one time. A 64-bit skyrim could in theory load every conceivable mod imaginable.

Steam workshop already makes mod support easy for devs and mod creators to distribute and insatll. So the need to break out from the 32-bit wall is already necessary.

http://www.gameranx.com/img/12-Aug/fpsmapdesign2010063628.jpg

This image is commonly used to deride modern FPS shooters. yet part of the reason for this is 32-bit architecture. Consumers DEMAND more polygons, more textures, more STUFF. Yet the memory contraints mean that you really have a lot of limitations on what you can do. FPS creators have to funnel you into areas because otherwise the engine would implode upon itself. Creators have to take the 2GB limit and figure out what they can and cannot possibly render at one time. Models, textures, lighting, etffects etc.

The move to 64-bit removes these barriers. It also means more memory can be allocated to other functions. Before you could never have the AI take up 1GB of memory. With 64-bit that's a reality and more.

Not sure whare you got that from but 64-bit is 64-bit thru and thru. As long as you compile the exe for 64bit.

Saroru1, I did say "I'm probably wrong"  and my thinking goes back a few years when game magazines were more commonplace and games were available with patches that had '64bit extensions'.

Great thread. This started back in December of last year. 

Does anyone want to say, there is no need for 64bit now that we have seen and had chance to play? Further up the thread there was mention about this game having almost no chance of hitting the 4 gig threshold in data. I am wondering if it has already passed it and if the new ship designs as previously stated do in fact add to the overall total used?

The coming 'Immense' and the 'OH MY GOD ITS GOT STARS IN IT' map sizes,  I am sure will test the memory limits. 

suprisingly GCIII doesnt use much ram a large map only utilizes about 2.7GB. Where the limits are pushed is vram, the game is sitting over 3.3GB usage(on my 4GB 770GTX). i think the next card races will be on vram and making them fast enough to process the available memory at faster rates along side with providing much more vram on cards which are released. 8GB Vram would allow for some breathtaking vistas to be created. Its great to see Stardock building their infrastructure on 64bit computing. The real innovators have always been the smaller dev/publishing houses.  

Before we extol the virtues of 64-bit we have to consider potential drawbacks

1. Optimization:  Developers had to optimize and hoard system resources to not hit the 2GB limit. Lets wait for the next EA hype hit that is poorly coded and will crash and hog all the ram and never release it.

2....Thats about it