Frogboy Frogboy

Lua vs. Python

Lua vs. Python

In previous journals we’ve discussed the intention to create a modding layer that would use Python as its scripting language.  Python is slower than Lua in terms of interpreted languages but there’s nothing stopping us from having C++ source that people could use to in addition or instead of a scripting language for performance sensitive elements of a game.

With that said, let’s hear from those familiar with both on what they think of either one and why.

405,000 views 157 replies
Reply #51 Top

As python programmer, I definitely prefer python over lua.

Whitespace is not a real issue if you follow PEP 8 and other guidelines. Keep code for each method reasonably short and flat, and you will never have indentation-caused errors.

(Yes, Python can do the same, but the whitespaces are even more of an issue when the goal is simply to represent data.)

Thats quite weird argument. Esp. considering the fact that brackets have own whitespace handling rules, which allow you to write EXACTLY same code as you presented:

Code: vb
  1. def translate(original_string):
  2.     return original_string
  3. monster = []
  4. monster.append(dict(
  5. name=translate("Dragon"),
  6. level=10,
  7. strength=20,
  8. description=translate("This dragon is called Dragon."),
  9. chat=(lambda self, who: who.allegiance == "Empire" and translate("I like you.") or translate("I don't like you."))
  10. ))
  11. print monster
  12.  
  13. #> python ~/tests/data_ewom.py
  14. [{'level': 10, 'strength': 20, 'name': 'Dragon', 'chat': <function <lambda> at 0xb7357dbc>, 'description': 'This dragon is called Dragon.'}]

The threading will likely be the application calling scripts from multiple thread rather than the scripts actually doing multi-threaded work on their own

I doubt Lua or other script interpreters could do that w/out using separate VMs for each C++ thread, or sticking to single thread per time with locks.

Reply #52 Top

Quoting marlowwe, reply 47
For those of you suggesting C++....are you serious? Do you not understand how unfriendly this language would be to someone who has little or no prior modding/programming experience?

Given Elemental's....unique...release circumstances the last thing it needs is C++ as its modding language. Whatever language is chosen, it needs to be *accessible* and easy to use. Python and LUA fit this requirement perfectly. Modding may be the one thing that might change Elemental's negative perception over time. Burdening the already thinned fanbase with learning C++ would seem like some sort of sick joke.

Also, it would be nice if we could get some concrete examples of how much of the game players will be able to change with modding. This might give us a better idea of what language is best suited for modding.

As long as one does not manually manage pointers* and more problematic/advanced features (templates**) it is not significally harder that a scripting langauge. If they are doing something that requires more that the basic features, then they will need C++ anyways.

* There are libraries that help with this. Lots of them. Even some that provide a garbage collector like those in scripting langauges.

** Templates are only bad when you do something wrong, at which point the error messages you get turn to gibberish.

Reply #53 Top

Quoting DKL, reply 51

I doubt Lua or other script interpreters could do that w/out using separate VMs for each C++ thread, or sticking to single thread per time with locks.

Lua uses separate VMs (or effectively does; they do not share data).

AngelScript on the other hand: http://www.angelcode.com/angelscript/sdk/docs/manual/index.html

Whitespace is merely objectionable; it is dynamic typing that is the real evil.

Reply #54 Top

 While Lua and Python have a similar solution to muliple threads the difference in implementation is pretty telling in how Lua was rewritten and redesigned from the bottom up with an API geared to manageable muliple threads and states. For the middle ground of multithreading there are the coroutines that generalize the concept of subroutines.

 As for dynamic an static typing I'm pretty much undecided. Or rather unimpressed by the weak safety guarantees offered by Java/C/C++ type systems as compared to strong type systems as offered in Haskell and logic programming langauges.

Reply #55 Top

Quoting Anangara, reply 54
As for dynamic an static typing I'm pretty much undecided. Or rather unimpressed by the weak safety guarantees offered by Java/C/C++ type systems as compared to strong type systems as offered in Haskell and logic programming langauges.

It is enough to deal with most mistakes. Why any compiler would not complain if you are attempting to multiply text by an integer is beyond me (unless you have taken special action to override safety, in which case you are asking for it to blow up in your face).

Reply #56 Top

Quoting Gwenio1, reply 44


However better multithreading will make it easier for the application to keep the scripts from messing up because of it. The threading will likely be the application calling scripts from multiple thread rather than the scripts actually doing multi-threaded work on their own.

Well that makes sense; but I also say it does not support Python well, ESPECIALLY with the global lock. 

My question then becomes why would this need to be managed at the script rather than program level?  Python (or any chosen language) would be a layer removed from the actual multi-thread execution of the game; I had figured the architecture for being C++ thread management (actual executing game), the mods being small bits that feed data, get passed or cause threads to communicate.    In other words, the scripts just feed the application and thus can handle handing off multi-threading with tagging / calls.

Ok, NOW I get why Gwenio1 is so hating on dynamic typing.  :annoyed:   This could make running multiple mods by different authors (especially non communicating ones) really painful. 

Anyways, I still maintain that easier learning curve of syntax languages for general interest modders vs grammar based ones.  Otherwise wouldn't we all be working in LISP?

Reply #57 Top

I OTOH don't get the point he's trying to make. He is using the names dynamic and static typing while describing the differences between weak and strong typing. Static typing is typechecking in the compiler phase. Dynamic typing is typechecking done in the runtime. Different concepts alltogether. Having programmed in assembler and Forth I can say it is a whole different animal.

Dynamic typing actually makes using different modules easier mainly because of the difficulty of doing a traditional static typecheck on unknown code hence defaulting to implementing assertions, invariants, typeguards and/or interfaces to cope.

Reply #58 Top

Have you thought about using something along the lines of http://thrift.apache.org/ which allows you to bind several languages against a common API.. It is a great way t allow a diverse range of languages which I think would give it a broader appeal!

I'd be happy with either LUA or Python, all languages are the mostly the same to learn anyway.. as long as c++ support is in there to get to the low level implementations. I'd really love to have access to write my own AI's for the computer players. Atm I like watching the AI play with Ctrl-Z on rediculous (except for the AI bug which is in atm which means only 3-4 of 10 AI's get off the ground as they run out of pop and then have  brain fart).

My 2 cents!

Reply #59 Top

AngelScript on the other hand: http://www.angelcode.com/angelscript/sdk/docs/manual/index.html

I believe you wanted to link http://www.angelcode.com/angelscript/sdk/docs/manual/doc_adv_concurrent.html this. Yeah, separate contexts is basically same as separate VMs, just lesser overhead. Point is, script data is not shared between threads, which significantly limits what you can achieve with scripts.

easier learning curve of syntax languages for general interest modders vs grammar based ones

All sane languages are "grammar-based". And, generally, "syntax" IS "grammar". As for learning curve, whitespace-based syntax is not harder than curved-brackets one. I've trained many people both familiar with programming and ones who had no prior programming experience to use python and syntax was never a problem. Many institutes like MIT switched to python as their introduction language because its easy to learn, yet very powerfull.

 

Reply #60 Top

Quoting Anangara, reply 57
I OTOH don't get the point he's trying to make. He is using the names dynamic and static typing while describing the differences between weak and strong typing. Static typing is typechecking in the compiler phase. Dynamic typing is typechecking done in the runtime. Different concepts alltogether. Having programmed in assembler and Forth I can say it is a whole different animal.

Dynamic typing actually makes using different modules easier mainly because of the difficulty of doing a traditional static typecheck on unknown code hence defaulting to implementing assertions, invariants, typeguards and/or interfaces to cope.

I am discribing both without bothering to switch terms; a variable in Python can be given text to hold and you can use the variable to do math in the code as you would a number (dynamic typing). The code will fail at run-time (strong typing). The first part is also true in Lua; however it will try to turn the text to a number and if it crashes or not depends on the text (weak typing).

C++/AngelScript will not allow the first unless you insert a command (called a cast) telling it you really meant to do so (it will still error if how the cast should be preformed is unknown). This is static typing.

I am assuming we will be distributing the source for script, as such static checks will be viable.

I can tell that you understand enough that I do not need to provide much of this info; it is for the benifet of others who may or may not need the extra info.

Reply #61 Top

I have gone through most of these posts and find there is a lot of support on all sides. It is interesting to see the various points for each side in this. My thoughts are fairly straight forward here, as I think the mod system should eventually be a three part system. The first part, which we already have, is the XML data section, where new content is added using a data driven methodology. The second part is the scripting part, and could be done effectively in either Python or Lua. The final part would be the DLL part or the C++ part. Each part has a significantly higher learning curve than the previous, which is why having a scripting section is so crucial. If the only alternative to the XML section was a C++ system, the learning curve for potential mod teams would probably be so high that only teams with experienced coders would be able to use this C++ system. This is not to say that it is not possible to layout the base libraries in such a way as to make them accessible, but C++ has a way of being unforgiving to the novice programmer in ways that a scripting language is not.

Now, as for the whether to select Python or Lua for the scripting language, I personally would select Lua. Looking at the differences between Python and Lua, a few particular pros for Lua are striking and are the reason for my support. Firstly, Lua has fewer keywords than python, and all of them are used for very simple and easy to learn syntax statements. Lua has 21 keywords, where Python has 31, including things like yield, lamba, and try, and a whole host of additional identifiers for specific classes. Secondly, the Lua syntax is fairly simple and does not force a rigid conformity in the ways that Python does. As has been stated previously, blocks in Python use the convention of indentation to determine blocks where as Lua uses a end statement which allow for a broad set of coding styles. Lastly, Lua uses a much simpler form of duck typing than python, which again allows for a much easier entry for the novice coder who may not be aware of all the intricacies of Python's duck typing implementation. All of these pros for Lua would make it is easier to learn, easier to code, and simply easier for the average user to enter into the modding arena.

Reply #62 Top

Quoting DKL, reply 59

\I believe you wanted to link http://www.angelcode.com/angelscript/sdk/docs/manual/doc_adv_concurrent.html this. Yeah, separate contexts is basically same as separate VMs, just lesser overhead. Point is, script data is not shared between threads, which significantly limits what you can achieve with scripts.

Actually it was: http://www.angelcode.com/angelscript/sdk/docs/manual/doc_adv_multithread.html

Data can be shared, but modifying it can be dangerous unless some means of locking (at a local level -- as in just that peice of data -- is availibe). In other words it is basically the same as it is for multi-threaded application written in C++ and other such langauges.

Reply #63 Top

a variable in Python can be given text to hold and you can use the variable to do math in the code as you would a number (dynamic typing). The code will fail at run-time (strong typing).

Well, its a bit different story for python. It does not care about 'types' at all. There are no types, everything is objects. Whether some binary operations can happen with a given set of objects and what actually happens is decided by objects themself in runtime, by calling appropriate object methods, so you can have strings which will be able to be used in math operations (moreover, 'some text'*10 is actually valid by default, for example, and will produce 'some textsome textsome text...'):

Code: vb
  1. >>> 'a' + 3
  2. Traceback (most recent call last):
  3.   File "<stdin>", line 1, in <module>
  4. TypeError: cannot concatenate 'str' and 'int' objects
  5. >>> class ForgivingString(str):
  6. ...     def __add__(self, a):
  7. ...         return str.__add__(self, str(a))
  8. ...
  9. >>> a = ForgivingString('a')
  10. >>> a
  11. 'a'
  12. >>> a + 3
  13. 'a3'
  14. >>> a + None
  15. 'aNone'

Reply #64 Top

Quoting DKL, reply 63

Well, its a bit different story for python. It does not care about 'types' at all. There are no types, everything is objects. Whether some binary operations can happen with a given set of objects and what actually happens is decided by objects themself in runtime, by calling appropriate object methods, so you can have strings which will be able to be used in math operations (moreover, 'some text'*10 is actually valid by default, for example, and will produce 'some textsome textsome text...'):

The point is invalid (or undefined) operations will fail. And I am sure player will be pleased when the amount of gold they have is a bunch of emotes. :-"

Oh and:

Code: c++
  1. inline std::string operator+(const std::string& s,int x) {
  2.   return (std::ostringstream(s.c_str()) << x).str();
  3. }

Should achieve the same as your sample. (Operators can sometimes be a bit diffrent in overloading, and it is not something you do often; therefore it might not as I do not feel like checking)

Reply #65 Top

Python gets my vote. I've been dabbling in Python in the past year and it has rapidly become my favourite programming language. Having used MATLAB for many years, I found that with the help of a few libraries*, Python can do everything MATLAB can do (e.g. easy matrix computations), while doing a lot of other things better (e.g. object oriented programming). For non-programmers, the syntax is about as simple as you can get, but programmers can do basically anything that you could do with C++ (and in fact you can embed C++ code in Python using weave - http://www.scipy.org/Weave).

* Useful libraries: Numpy, scipy and matplotlib - http://www.scipy.org/, http://matplotlib.sourceforge.net/. Windows users can just download Python(x,y) (http://www.pythonxy.com/) for a complete MATLAB replacement in one package.

Reply #66 Top

Quoting kenata, reply 61
...My thoughts are fairly straight forward here, as I think the mod system should eventually be a three part system. The first part, which we already have, is the XML data section, where new content is added using a data driven methodology. The second part is the scripting part, and could be done effectively in either Python or Lua. The final part would be the DLL part or the C++ part. Each part has a significantly higher learning curve than the previous...

... All of these pros for Lua would make it is easier to learn, easier to code, and simply easier for the average user to enter into the modding arena.

You may have a point. If we are given access to C++ on top of it all, then perhaps the easier of the two languages would be better as an intermediary step. It seems like Python is almost used as a replacement language to C++ based on how powerful it is. On the other hand, if someone is going through all the trouble of building a mod, is the difference in complexity between the two languages really that appreciable? It seems kind of small compared to the scope of actually building something with either language.

I'm standing on the fence with this one. Not that my opinion matters, but I find the discussion interesting. I will be happy with either one.

Reply #67 Top

You may have a point. If we are given access to C++ on top of it all, then perhaps the easier of the two languages would be better as an intermediary step. It seems like Python is almost used as a replacement language to C++ based on how powerful it is. On the other hand, if someone is going through all the trouble of building a mod, is the difference in complexity between the two languages really that appreciable? It seems kind of small compared to the scope of actually building something with either language.

I feel that programmers can easily lose perspective on the sheer difficulty of picking up a new system and learning both the syntax of the supported language as well as the intricacies of the system.  These are skills that most programmers pick up as a matter of their trade, yet programmers are not the only people who want to do mods. In fact, many mods are done without the support of an actually trained programmer at all, and so this is something that should be well considered. Python is a very powerful scripting language, and like a previous poster pointed out, has libraries to perform many specialized functions. Yet, this power comes at the cost of simplicity. It is funny that you describe someone deciding to undertake a mod as "Going through all the trouble of building a mod." A good modding system should not be considered as "Going through all the trouble" for the vast majority of modders. While a small amount of a mod community may be teams of individuals creating new art assets and new code, the larger majority are simply individual modders looking to change a few game mechanics or add some new content for their own enjoyment. Yet, I think you have a point. If we are talking about a 3 tiered system, a scripting language like Python adds almost nothing when one has access to a C++ DLL. However, something like Lua trades off overall power, which we have already pointed out is not traded but moved to a more appropiate system, for overall simplicity of coding.

Reply #68 Top

Quoting Gwenio1, reply 38

Once you have learned one langauge, learning the others is not that hard.

And learning a langauge is the easy part if you are going to do more than simple data manipulation.
It's not a matter of easy vs hard but of "We were told Python, devoted some time to (try to) learn (and somehow master some of) it when in normal circumstances we wouldn't (, because I should be wasting my free time with Java for supposed new job opportunities), and now we are told that we may end not using it.". I don't really care one or the other as long as it benefits the game and surely Python/C++ for Civ IV seemed to do great. Yet Civ V went Lua (which may or may not mean anything). O_o

Reply #69 Top

I don't really care one or the other as long as it benefits the game and surely Python/C++ for Civ IV seemed to do great. Yet Civ V went Lua (which may or may not mean anything).

Civ IV is what i attempted to learn python for.. after hours and hours of getting no where with it i pretty much gave up on modding anything in civ IV outside of XML.  Its why i dislike python as a language.. it was harder for me to learn than any other scripting language i've personally dealt with. But then again i was pretty much spoiled with Biowares in house scripting langauge for NWN. lol. You can pretty much blame my  inability to learn more complex scripting languages on them :thumbsup:

 

I guess my opinion is pretty much moot on this point to be honest.. cause i don't honestly see myself attempting to struggle learning  python to mod elemental. I'd much rather learn C++  if i gotta deal with a complex language. Although I would be willing to atleast attempt to learn Lua.. i've already personally written python off after my civ IV struggles.

 

 

Reply #70 Top

I have fairly advanced knowledge of Python, and passing knowledge of Lua.  I admit that somewhat informs my vote, but my vote is quite firmly with Python. Three reasons for this: First, Python is as far as I can tell the king of multiparadigm.  I have yet to see a programming paradigm that doesn't work at least decently in python (although, admittedly, a jack-of-all trades is often a master of none...), and that is appealing in a binding that will go out to a bunch of people that will have projects likely ranging from the tiny one-or-two function modules to an entirely different game that just happens to use the same engine (for which e.g. Object-Oriented Programming is likely to be more appropriate).  Of course, that may mean the bindings to Kumquat have to be multiparadigm also, but that shouldn't be too hard once there's at least a low-level API that can be built from.  Second, I find teaching Python to inexperienced or non-programmers (of which many modders are) to be a generally much easier task than teaching other languages (although that's partly because of reason #1).  Third, my perception is that the Python community is much larger than the Lua community (although the difference is somewhat less if you limit it to just the gaming/modding community), and that allows more opportunities for more people to join in the fun.

One other thing:  there has been a fair amount of discussion about multithreading... First of all, I found it doubtful most modders will be inclined to use any extensive multithreading given how buggy it is if you're not very careful (as Frogboy's AI posts have frequently noted!).  So I think overall it's a fairly moot point.  But even so, depending on how the scripting is written, it is most likely straightforward to get around Python's GIL limitations by just running another process, either by designing the bindings to the Kumquat engine that way, or using the multiprocessing module directly. In my experience, this is actually much *easier* to write than multithreaded code, because you don't have to worry as much about locking access to various objects (processes generally have to share via messages or pipes instead of actual objects).

P.S. While Frogboy seems to be wanting a scripting language, I'm sure we're all actually hoping that if they can't find a Fortran 77 compiler as someone else in the thread suggested, Stardock will instead opt to go with COBOL.

Reply #71 Top

I've used both languages in one capacity or another recently, and my personal preference would be Lua. Both languages are comparatively easy to write in, but Lua seems to lend itself better to embedded scripting, whereas python is utterly fantastic for standalone scripts/tools/anything running under the OS/shell scripts rather than trying to plug into a game. 

For background I've been writing game code (not mods, it's a mix of gameplay scripting, tools, and engine code) with a games company for several years now, and we've found that lua is:

 - Quicker for existing C/C++ programmers to learn than Python

 - Lower memory footprint (as mentioned before)

 - Can have a much more predictable garbage-collection profile. It's subtle but I've seen more minor framerate spikes & stutter when using python code than Lua, that being said the majority of it can be mitigated with smart code in either language.

 - Relatively easy to edit for less technical designers, at least for basic tasks.

 

One big problem in either language though - debugging & profiling. How is stardock planning to handle this? and are modders going to get access to the tools? I'm sure we've all tried writing complex code without a debugger before (and probably dumped mountains of trace statements in to find out where things are going wrong), but it's *always* a pain - time spent to invest in at least a simple lua debugger is very well worth it, even if it's only to connect after hitting a breakpoint/error (or be an in-game UI) and examine the current state.

 

And to the people requesting that C++ is used, do you *really* want to be in a situation where a rogue/malicious mod has the full privileges of the game to do things to your system? It's very hard to properly sandbox native code - possible, but hard, and is it really worth Stardock spending time on that for this game, rather than spending time on the game itself? Whether intentionally or accidentally mods written than way would be far more likely to corrupt/crash the game, or play merry hell with your system - I'd much prefer a nice isolated scripting language that has absolutely no way of calling outside it's well defined API, and doesn't have opportunities for all the funky pointer math that could screw up said sandbox.

Reply #72 Top

Non programmer here

- Relatively easy to edit for less technical designers, at least for basic tasks.

And this has to be the winner.  You want something that will encourage new moders.  Apparently Lua has better handling of data - it is easier to adjust all the tech costs by a factor in Lua (so says the Civ V modding documentation anyway). 

Quite a few mods will be centred around adding to or modifying the data tables. (new buildings, new units, new quests, new spells).  You want to make that sort of stuff as easy as possible.  That may be through some form of XML interpreter (sombody mentioned spreadsheets for Fallen Enchantress - that hits my comfort zone), but it may be Lua could handle that bit as well.

 

Reply #73 Top

Lua has always been an excellent data description language as well. Its ancestry is SOL (Simple Object Language) and DEL (Data Entry Language) and usage in commercial petrochemical and geological data processing and configuration.

Reply #74 Top

One thing to consider, in a post World of Warcraft era, many more modders already know Lua than know Python (Civ 4 being the biggest major moddable title that used Python that I am aware of - even Civ 5 switched to Lua).

Reply #75 Top

For those of you suggesting C++....are you serious?

My thoughts exactly: it is a tough language, plus isn't exposing the source code a little dangerous?  Modders could do some serious damage to their Elemental install or at the very least release mods with serious memory leaks.  Although I prefer Lua, I'd have to say Python is the way to go as it is simpler and modding should have as a low a barrier to entry as possible.  Well written Lua code doesn't happen quickly as it really is different from standard OO languages and tables take some getting use to.  Also total agreement:

And to the people requesting that C++ is used, do you *really* want to be in a situation where a rogue/malicious mod has the full privileges of the game to do things to your system?