PDA

View Full Version : UndeadScript matures (slowly, but it does :-)


Bad Sector
10-10-2006, 02:40 AM
UndeadScript is the scripting language i started writing at some point for my Undead Engine (now Incomplete Undead Engine, since some months ago i started to rewrite everything from scratch but i wanted to keep the "Undead Engine" name and release the source code). It wasn't really completed and i was doing it mostly as an experiment to try making a strong typed scripting language (my previous effort - NerveBreak, which i abandoned at 2005, there is a SourceForge.net project if anyone wants to take over and continue the development - was typeless to the maximum: assign a number to a variable, use it as a string and BOOM).

However it evolved quite nicely. I even used it in Nikwi (http://www.slashstone.com/more/nikwi) to script the monster and item behavior, although the version i used there was very primitive (no function support, for one). These days i needed a scripting language to be around and since i have a "i do everything myself" mentality, i started working on UndeadScript again. I would need to work on it, anyway, once the engine came to a usable state, so the sooner the better because i can use it in other projects aswell (it is not bound to the engine, although it uses the "Undead" part and the "U" prefix in classes).

Up to this point i managed to put proper function support, which -unlike in NerveBreak- don't mess up their variable's data if they call themselves :-), language-aided export support so you can get the address of a function (in order to call it) or variable (in order to read or modify it) from the host program (the game engine), proper floating point and "handle" support (unknown pointer type - can be used for entity handles, etc without exposing their internal structure to the script), array support (a bit limited, though) and the last-and-best: dynamic allocated structures and reference-based garbage collection :-).

The last two means that you can allocate structures defined either by the script or by the host program (currently i haven't wrote the parser part for declaring structures, but it isn't something hard) and allocate memory for them in order to use them.

The garbage collection part means that you don't have to worry about the memory you allocate: once something is not required anymore is dropped and the memory it reserved becomes available again. The "reference based" means that the scripting engine keeps track of where an object (allocated memory for a structure) is used in order to perform garbage collection. For example, in the following code:


Vector v = new Vector;


the created object that results by "new Vector" has a reference count of 0. When assigned to the 'v' variable, it gets a reference count of 1. Extending the code:


Vector v = new Vector;
Vector z = v;


z has the same value as v, the object created by "new Vector" and now that object has a reference count of 2 since it is "referenced" by two variables. But if you extend it to:


Vector v = new Vector;
Vector z = v;

v = new Vector;


Now 'v' has a new object (of the same type) and 'v' and 'z' refer to two different objects. Since 'v' had a reference to another object prior the last line, that other object loses a reference. Now there are two objects and both of them have a reference count of 1. Lastly, if you extend it to:


Vector v = new Vector;
Vector z = v;

v = new Vector;
z = v;


Both 'v' and 'z' refer to the second object and since no variable is referring to the first object anymore, the first object has a reference count of 0 and becomes garbage which will be dropped by the garbage collector.

To be exact, however, an object with reference count of zero doesn't dropped immediately. This is done because it may exist somewhere inside the scripting engine's internal structures or be part of a 'return' statement (used in functions) or for other reasons lose temporary it's references. To deal with this situation, the scripting engine gives a "potentially garbage object" two chances to prove it is not garbage and every 1000 virtual machine instructions, it is checked for 'garbage-ness'. If fails in both chances, then it is dropped. I must note though, that i don't know if this is a "robust" method, but i did many tests and it never failed (in the sense that no object was deleted when it actually WAS needed or garbage object remained in memory). So i can assume that it is a safe bet to say that it 'works'. Of course these were prefabricated tests, i'll know if it truly works once i use it in a real game or program.

For the future, i plan to put true objects and classes in order to make it an object oriented scripting language. It will be very helpful to create classes for -say- monsters where generic monster behavior is scripted in one abstract Monster class and monster-specific behavior is scripted in specific classes derived from that class. And i since i have functions, structure objects and garbage collection, i don't think it will be much of a problem to put classes in there.


But anyway, for the moment, here is a little script showing UndeadScript's syntax (below are the results of running this script):

/*
** UndeadScript math test
*/

Vector originVector()
{
return new Vector;
}

Vector createVector(float x, float y, float z)
{
Vector v = new Vector;
v.x = x;
v.y = y;
v.z = z;
return v;
}

float vectorLength(Vector v)
{
return sqrt(sqr(v.x) + sqr(v.y) + sqr(v.z))
}

float vectorDot(Vector a, Vector b)
{
return a.x*b.x + a.y*b.y + a.z*b.z;
}

Vector normalizedVector(Vector v)
{
float len = vectorLength(v);
return createVector(v.x/len, v.y/len, v.z/len);
}

Vector vectorCross(Vector a, Vector b)
{
return createVector(a.y*b.z - a.z*b.y, a.z*b.x - a.x*b.z,
a.x*b.y - a.y*b.x);
}


dumpVector(createVector(1.0, 2.0, 1.5));
dumpFloat(vectorDot(createVector(1.0, 2.0, 1.5), createVector(1.0, 0.3, 0.4)));

Vector n = normalizedVector(createVector(1.0, 2.0, 1.5));
dumpVector(n);
dumpFloat(vectorLength(n));


Vector z = normalizedVector(vectorCross(createVector(1.0, 0.0, 0.0),
createVector(0.0, 1.0, 0.0)));
dumpVector(z);

(ps. click here for a syntax highlighted version (http://www.slashstone.com/gimme/test1.us.html))

And the results (copy/paste from the console):

badsector@terastios:~/projects/us$ ./usdemo
x=1.000000, y=2.000000, z=1.500000
2.200000
x=0.371391, y=0.742781, z=0.557086
1.000000
x=0.000000, y=0.000000, z=1.000000
badsector@terastios:~/projects/us$

electronicStar
10-10-2006, 05:40 AM
Awesome! I am now officially following this project.
I was disgusted of Unrealscript because of the price of a license to the unreal engine:o
I hope your engine price will be much more indie friendly, you'll have at least one customer for sure:cool:

Bad Sector
10-10-2006, 06:03 AM
My engine's price will be very indie friendly: free :-P

I don't really seek to gain money by developing middleware (except that at some point i may create a game-creator thing, but this is something that i only have in my mind and it's not in my short-term plans). I prefer to games for that :-D.

I posted it here mostly because i'm excited that it works. I tried to make a similar script engine design (with different syntax) for SlashBASIC, but i had dozens of problems. Once i managed to fix them, my hard disk crashed and lost the sources :-(. In UndeadScript's case, i used a similar (but not equal) design, but things are coming much better now - i suppose due to the added experience i had with SlashBASIC and knowing what to avoid :-/.

Anyway.

I'm gonna use UndeadScript for my own games and programs. But just like some other stuff i wrote, i think that giving the sources out may help some other developers.

Besides, giving it for free is a nice way to give up any official support :-P.

newobj
10-10-2006, 07:42 AM
hmm, no offense, but practically speaking, what would this offer over any other existing scripting language, e.g. lua, javascript, python, ruby, etc... ? it still seems awfully low level.

secondly, garbage collection by reference counting does not deal with circular references. e.g.,

Node n1 = new Node;
Node n2 = new Node;

n1.next = n2;
n2.next = n1;

once n1 and n2 fall out of scope, there are no reachable references, however, they both have ref counts of 1 and will not be garbage collected.

will your system be able to deal with this?

Backov
10-10-2006, 08:46 AM
Seriously, don't develop your own scripting language.

There are many languages that would work much better right out of the box, and that you wouldn't have to write. Your users, if you ever have any, won't thank you for it. They'll wonder why they have to use some NIH language instead of a more standard language like Python or LUA.

If you ever plan on getting anything done, don't do it. Trash what you have and integrate a prebuilt scripting language. For your NIH sickness - seek therapy.

Bad Sector
10-10-2006, 08:47 AM
hmm, no offense, but practically speaking, what would this offer over any other existing scripting language, e.g. lua, javascript, python, ruby, etc... ? it still seems awfully low level.

Well, to be honest, i don't know. From the mentioned languages, i've only used JavaScript (i'm doing very advanced JavaScript work in my day job) and i can tell that it can't be used efficiently with games (please note: i'm not saying that it can't be used at all - it just gives me the impression that it will be slow when compared to a strong typed language). Also i wrote a little Python, but it wasn't enough to say that i know the language.

The only thing that i can tell you is that personally i dislike their syntaxes :-). From these, only JavaScript has a sane syntax. Keep in mind that don't consider anything that doesn't look like C to be "high level" (in that manner, COBOL should be a very high level language :-P). It can look like C and be high level. And personally, i like C.

secondly, garbage collection by reference counting does not deal with circular references. e.g.,

Node n1 = new Node;
Node n2 = new Node;

n1.next = n2;
n2.next = n1;

once n1 and n2 fall out of scope, there are no reachable references, however, they both have ref counts of 1 and will not be garbage collected.

will your system be able to deal with this?

No. There are issues with every garbage collection system. I used reference counting only because it is fast when compared to other methods i tried and speed is one of the key things i want in UndeadScript.

Currently i don't consider this to be a major issue.

Bad Sector
10-10-2006, 08:51 AM
If you ever plan on getting anything done, don't do it. Trash what you have and integrate a prebuilt scripting language. For your NIH sickness - seek therapy.

Hey, i like doing that stuff, i don't consider it sickness or whatever bad :-).

Backov
10-10-2006, 09:21 AM
Hey, i like doing that stuff, i don't consider it sickness or whatever bad :-).

Oh I can totally dig it man. But it's not a good thing for productivity, and as I said, it's serving no purpose but to make you happy. It's not going to make your users happy. It's not going to make your engine easier to use or less buggy - in fact it will probably make it worse on both counts.

If you ever want to get anything done, you need to overcome NIH. Are you going to invent your own audio library when OpenAL is not to your taste? Your own input library because you don't like the method names of XInput?

HairyTroll
10-10-2006, 10:58 AM
The only thing that i can tell you is that personally i dislike their syntaxes :-). From these, only JavaScript has a sane syntax.

What does your new scripting language provide that existing scripting languages do not? Is it just more of the same but with a slightly different syntax?

impossible
10-10-2006, 12:47 PM
The only thing that i can tell you is that personally i dislike their syntaxes :-). From these, only JavaScript has a sane syntax.
Huh? Although you can personally dislike anything that doesn't look like C++ or Java, saying Lua, Python or Ruby don't have sane syntax is pushing it... In my opinion (and many others) working with these languages is very nice. Lua is more than fast enough for game scripts, and is heavily used in a variety of high profile games (Psychonauts, WoW, Homeworld, Dawn of War, FarCry, Crysis, etc.)

I have no problem with NIH, if you want to rewrite Java go ahead. But not all of the off the shelf languages out there are too slow or ill-suited for games. Your language might be faster than Lua is, but you also need to consider how productive your language is. With scripting, if something really needs to be done fast it will be moved to C++ anyway. The only advantage I see with your language you give script coders a sandbox environment and faster iteration times. Maybe you're planning on adding game and engine specific constructs. If that's the case you shouldn't have to write your own vector class, it should be a built in type.

You might still want to consider integrating Lua into your engine. It should be a pretty quick process, and you might like the results.

Btw, Python uses reference counting but it also has an (optional) cycle detector.

Bernard François
10-10-2006, 02:18 PM
If Bad Sector wants to be productive, he's probably not doing the best (or at least the easiest) thing. But since he's not making a living from it, he might use the experience later in his career to do something more productive. I don't consider it a waste of time.
Also, if he decides to give up on his scripting language and he makes it open source, the language might continue evolving until it gets something really good (which can be a matter of years).

About the reference counting... I don't think the way you handle reference counting is a good idea, since it doesn't theoretically (and thus practically) excludes wrongly deleted objects. You should try to count every reference, even the ones created when your object got assigned to a variable after a return.

Bad Sector
10-11-2006, 01:37 AM
But it's not a good thing for productivity, and as I said, it's serving no purpose but to make you happy. It's not going to make your users happy. It's not going to make your engine easier to use or less buggy - in fact it will probably make it worse on both counts.

Uh, what's so hard understanding that i'm not only after finishing the game, but i like the process of writing the engine and all the subsystems?

If my goal was to have the game made as quickly as possible using as much existing code as possible, i would purchase Torque. It includes a 3D engine, a scripting language, a world and gui editing tool, can work with QuArK world editor -which is a world editor i know well- and seems to have good support.

If you ever want to get anything done, you need to overcome NIH.

When my goal is the result, i try to use everything i already have available.

Are you going to invent your own audio library when OpenAL is not to your taste? Your own input library because you don't like the method names of XInput?

I haven't got to a point where SDL's sound functions (hey, look! i didn't reinvent SDL -- yet) aren't enough, but if OpenAL or XInput isn't my taste, then why not? As i stated above, i like doing that stuff.

What does your new scripting language provide that existing scripting languages do not? Is it just more of the same but with a slightly different syntax?

Yes.

Huh? Although you can personally dislike anything that doesn't look like C++ or Java, saying Lua, Python or Ruby don't have sane syntax is pushing it... In my opinion (and many others) working with these languages is very nice. Lua is more than fast enough for game scripts, and is heavily used in a variety of high profile games (Psychonauts, WoW, Homeworld, Dawn of War, FarCry, Crysis, etc.)

I said that personally i think that. It's a subjective thing.

I have no problem with NIH, if you want to rewrite Java go ahead. But not all of the off the shelf languages out there are too slow or ill-suited for games.

I never said that.

Your language might be faster than Lua is, but you also need to consider how productive your language is. With scripting, if something really needs to be done fast it will be moved to C++ anyway. The only advantage I see with your language you give script coders a sandbox environment and faster iteration times. Maybe you're planning on adding game and engine specific constructs. If that's the case you shouldn't have to write your own vector class, it should be a built in type.

Vector is a built in type. I was just checking the compiler's sanity (it isn't yet finished, you know...) with the above functions. When it's finished, the engine will have some built in classes to work with.


You might still want to consider integrating Lua into your engine. It should be a pretty quick process, and you might like the results.

Btw, Python uses reference counting but it also has an (optional) cycle detector.

I don't like Lua. Python is ok-ish, although i dislike the language's "feature" to use indentation for scope.

If Bad Sector wants to be productive, he's probably not doing the best (or at least the easiest) thing. But since he's not making a living from it, he might use the experience later in his career to do something more productive. I don't consider it a waste of time.

Well, actually my ability to write language parsers (i'm doing this since ~1995) was one of the pluses that got me the job i have now :-P. At the other hand, i can't say i really like my job, but well, it's better than burger flipping....

About the reference counting... I don't think the way you handle reference counting is a good idea, since it doesn't theoretically (and thus practically) excludes wrongly deleted objects. You should try to count every reference, even the ones created when your object got assigned to a variable after a return.

I do count every reference, including those created when objects are assigned to variables after a return. However doing an explicit "check" for that case and any related case would be extremely complex since there are many cases where this can go wrong (and i may forget some). I use the method i described (well, it's not the full description, so keep in mind that) in a few key spots and it seems to work well. The reference counting happens in assignment (decrement/increment), function call (increment) and function return (decrement).

Note again that this is a generic description, there are some fine details i may forget right now :-).

gosub
10-11-2006, 07:54 AM
I think it's great that you're writing your own scripting language. But you should at least understand (preferably use) most of the others first. Especially C#. The garbage collector in C# is fantastic - fast and no reference counting.

-Jeremy