More precisely, Eric Preisz and I wrote a book!

The book is called Video Game Optimization, and it covers everything you need to know to get maximum performance from any software project – but especially games. If you’re struggling with getting a great framerate out of your game, I highly recommend checking it out.

Video Game Optimization goes all the way from high-level concepts like planning for performance in your project’s timeline, to determining which broad area of your system is a bottleneck, down to specific tips and tricks for optimizing for space, time, and interactivity. Based on the course that Eric Preisz taught at Full Sail University on optimization, it isn’t the only book you’d ever want to read on the subject, but we think it is a great introduction!

The journey from that initial conversation where our mutual friend Jay Moore introduced us and suggested I would be a good co-author, to the day when we finished and shipped the book was a long but rewarding trek. Eric moved across the country from Florida to Nevada, as he moved from teaching at Full Sail University to running the Tech and Tools group at InstantAction. He also became a father with the arrival of his son, Grant. I left after 5 years at GarageGames and helped build a new company, PushButton Labs.

A lot has changed while we wrote it, but it still felt really good to arrive at GDC, visit the book store outside the exhibition hall, and finding a big stack of Video Game Optimization sitting front and center.

It is pretty cool stuff, and I’m excited to be sharing it! You get introduced to the PushButton Engine, get a preview copy of Clint Herron’s excellent Platformer Starter Kit, and (assuming things go smoothly), you end up building this platformer:

We’re excited about it.

If you are a Flash game dev and at MAX or in the LA area, I’d love to talk games with you. Shoot me an e-mail (ben dot garney at gmail dot com) or DM me at @bengarney and let’s make it happen!

Also, with luck, Monday there should be a cool update on that secret project I was working on.

See you at MAX!

Google released O3D, a web plugin for 3d rendering, today. It’s a pretty sweet piece of work. Definitely check it out if you have any interest in 3d rendering, the web, or JavaScript.

There are a couple of cool pieces to this puzzle, and I wanted to call them out to other people who might be reviewing this technology. The two sentence review: I am blown away. They got this right.

(FYI: This post isn’t meant to be a full walkthrough, just a quick indication of the cool jumping off points in the codebase.)

Stop number one is the graphics API abstraction. This is rooted in o3d/core/cross/renderer.h, which provides an abstract interface for performing render operations. This is sensibly designed, oriented for SM2.0 through SM4.0 level hardware. It will also deal with DX11 class hardware, although it won’t expose all of the niceties DX11 gets you. It wraps GL (o3d/core/cross/gl/) and D3D9 (o3d/core/win/d3d9). The purpose of all this is to provide a common ground for all the higher level code to issue draw commands from. It is not directly accessible from JavaScript (although some of the related classes like Sampler and State are.)

As an aside, writing a Pixomatic or Larrabee backend would not be hard. At GarageGames, we had a rendering API similar to this (unimaginatively called GFX), and it was fantastically useful. Last time I checked, there were DX8, DX9, DX10/11, GL, and Pixomatic backends for it in varying states of usefulness. We even had one guy write a backend that would stream draw commands over a socket, which was pretty cool. In the context of O3D, they have an implementation of Renderer that queues everything into command_buffer, then streams it to a command buffer server running in a separate thread, which issues the actual draw commands. It’s unclear whether this is used in the current version of the plugin based on casual inspection, but it’s a great example of what good design can get you.

The next piece is the DrawList, which is where most of the heavy lifting for drawing happens. JavaScript, of course, is not really desirable to have in your inner rendering loops, so you queue everything you want to draw (DrawElements) into a DrawList. This is wrapped by higher levels, of course, but it represents the lowest level API that’s available to JS code. All the sorting and state management to get stuff on screen happens in this area.

Alongside the DrawList stuff, you find the material system, which is pretty slick. You write your shaders in their shading language, and it converts to HLSL SM2.0 and Cg (arbvp1/arbfp1). This is enough to do almost anything you might want to (as fantastic as SM3.0+ is it’s not really necessary for most rendering). There’s a full SAS system so you can interface programmatically with your shaders.

Above this, you get into the scenegraph. Now, I subscribe to Tom Foryth’s views on scenegraphs, which is that they are basically a bad idea, but I think the Google guys were smart and set up their API so intelligent developers can avoid being screwed by the scenegraph. The scenegraph-esque stuff they do have (they call it a rendergraph) is powerful enough you can do most rendering without going nuts, and lets a lot of the heavy lifting stay in native code, where it will be fast. You end up doing retained mode-ish things, but since JS is slow, and most JS developers come from a DOM background, it works better than you might expect.

There are a lot of cool miscellaneous features, too. They embed the fast V8 JS engine right in the plugin so you can have consistently fast JS execution. They support falling back to an error texture (via Client.SetErrorTexture) when you fail to bind a sampler. You can group objects via Pack objects for easier management and stream things from archives, too. There are debugging aids and libraries for math and quaternions.

You should check out the samples. There’s a lot of impressive stuff. There’s the beach demo in the video at the top of this post, but they also do some cool stuff with animation, HUDs, text rendering, picking, and stenciled teapots. And everything is done in JavaScript right in the web page. Even the shaders are embedded in <script> tags!

O3D is a great piece of technology, and I hope it thrives. I’m excited to see what people build on this (and I’ve got a few ideas myself .

We tried a few approaches – hand-editing a large XML file (but it was too large and spread out) and an in-game tweaking UI (but it was too much work to get the UI to be friendly to use). The old standby of having the designer and artist file bug reports to have the programmer update the game wasn’t getting us very far, either.

“Well,” says I, “We’re some sort of Web 2.0 startup, right? And we’re developing a Flash game aren’t we? And Flash can talk to websites, can’t it? And don’t we use Google Docs for everything?”

It turns out there is an XML feed from public Google spreadsheets. And ActionScript 3 supports E4X, so you can directly manipulate XML without any extra work. Now we tweak our game using a shared spreadsheet up on Google Docs.

I wrote a parser for their format:

// Extract the entries. It's namespaced, so deal with that.
var xmlns:Namespace = new Namespace("xmlns", "http://www.w3.org/2005/Atom");
tweakXML.addNamespace(xmlns);

// Parse into a dictionary.
var cellDictionary:Dictionary = new Dictionary();
for each(var entryXML:XML in tweakXML.xmlns::entry)
{
   cellDictionary[entryXML.xmlns::title.toString()] = entryXML.xmlns::content.toString();
}

And wrote a quick component that would fetch the spreadsheet feed, parse it, and stuff it into the right places on named objects or template data. Now I have a little entry in our level file that looks like:

<object id="googleTweaker">
   <component class="com.pblabs.debug.GoogleSpreadsheetTweaker">
     <SpreadsheetUrl>http://spreadsheets.google.com/feeds/cells/0d8somekey848x/od6/public/basic</SpreadsheetUrl>
     <Config>
        <!--  Grunt Level 1 tweaks -->
        <_><Cell>B3</Cell><Property>#TheGruntProxy.creator.WarPointCost</Property></_>
        <_><Cell>C3</Cell><Property>!Grunt.health.mMaxHealth</Property></_>
        <_><Cell>D3</Cell><Property>!Grunt.ai.AttackSearchRadius</Property></_>
     </Config>
  </component>
</object>

Each line maps a cell in the spreadsheet to a property on a template or active game object. Some properties have to be set several places, which the system deals with automatically.

The biggest wrinkle was Google’s crossdomain.xml policy. Basically they do not allow random Flash apps to access their site. So I had to write a small proxy script, which sits on our development server next to the game and fetches the data for it. Figuring out I had to do this took more time than any other step.

The main difference between the snippet and the full code is the version in our repository is 220 lines long. I only have around 150 of the full set of 200 parameters hooked up, but after a hard afternoon’s work, the process for tweaking the game has become:

1. Open Google Docs.
2. Edit a clearly labeled value – like level 1 grunt health.
3. Restart the game, which is running in another tab in your browser.

This takes you about a minute between trials. Not too bad. Before this, the process was:

1. Get the game source from SVN.
2. Find the right XML file – there are several.
3. Find the right section in the XML – altogether we have 200kb of the stuff for Grunts!
4. Change the value.
5. Commit the change.
6. Wait 5-15 minutes for the build system to refresh the live version of the game.

Ten minutes per tweak is not a good way to develop.

I’ve heard about developers using Excel spreadsheets for tweaking, but can’t find anything about using Google Docs to do it. But Google Spreadsheet is obviously a better choice. It has built-in revision tracking. You can edit it simultaneously with someone else. You can access live data in XML either publicly (like we did) or privately via their authentication API. It’s absolutely worth the half-day of your time it will take to add Google Spreadsheet-based tweaking to your game – even if it’s a non-Flash game, downloading and parsing XML is pretty easy with the right libraries.

I strongly suspect this feature will find its way into the next beta of the PushButton Engine. Which, by the way, you should sign up for if you are interested in developing Flash games. We’re bringing people in from the signup form starting this week. If you want more information, or just like looking at cool websites, click below to check out the new version of the PBEngine site, which has a bunch of information on the tech. Tim did an awesome job on the site design.

Edit: Patrick over on the GG forums asked about the proxy script. It’s actually ludicrously simple. Not very secure either so I wouldn’t recommend deploying it on a public server. I got my script from a post on Aden Forshaw’s blog. In the real world you would want to have some security token to limit access to your proxy script… but since this is for tweaking a game that is in development I didn’t sweat it.

for those who may come across this post on their search to learning how to set up a command-line-only Flex environment, and who also know previously zilch about Flex/AS/Flash, i sketch the steps very roughly here: http://elenzil.com/flash/flash_1

– Orion

If you’re into Flash coding, you should also read Rob Sampson’s post on AS3 Math Optimization – int is the new floor(). Rob’s been doing ActionScript a lot longer than I have, and I love the historical perspective he can provide. You can tell he’s a graphic designer – his performance charts look awesome.

I have to quote this from Jeff Tunnell’s blog, Make It Big In Games:

I stumbled across this YouTube video of Gary Vaynerchuck of Wine Library TV giving a presentation at Web 2.0 Expo. At first I just thought the guy was a dick, and he even calls himself that at one point in the presentation, but it turned out to be a fascinating 15 minutes of video. Gary essentially says what I have been saying, i.e. make sure you love what you do, work hard, and things will work out, but he says it in a much more succinct, hard edged way than I do.

Check out the video here.

As I was filing them away, I looked back in my .plans at GarageGames and realized that I had never talked about it publicly. At least, the site search didn’t turn anything up. Well, I think it’s probably safe to do so now.

Way back in April of 2005, I wrote a Blitz3D loader in Torque for Adrian Tysoe. It loaded Blitz3D files, with all their texture and material data, and it did polysoup collision, as well. This was several years before I released the OPCODE-based poly soup resource, which I think is now (in some form) in the official Torque codebase.

I love seeing how graphics code comes together, so I thought I would share this with the world – you might like clicking through the images and seeing how each step changed things. By far the biggest pain point was parsing the (sometimes obtuse) Bitz3D format. The format might be a pain, but in the hands of a good artist like Adrian, it gave really solid results. This was all before shaders were mainstream, of course, but still – the sand had a nice specular highlight, there was lots of clever texture layering, and the whole thing was pretty fast to render.

The more interesting aspect to doing the B3D loader was the prevailing opinion in the Torque community at the time that loading a non-DIF/DTS format was impossible – as was deviating from the collision models those formats used (convex polyhedra mostly). I’m not quite sure where they got this idea, but it was awfully frustrating at times to see people beat their brains out on those formats when there might be better options for their specific situation.

Partially, I did this project to prove to myself that I wasn’t just bullshitting people when I said that writing a loader/renderer was easy (certainly easier than canceling your project because another format wasn’t working for you!). It took me a month or so of part time hacking to get it all together. It didn’t animate, but it was just for environments – DTS is actually an awesome format for characters, and the 3Space runtime is very tight, one of the best parts of Torque in terms of features. (Aside: I hope they get a good Collada->DTS converter as the main pipeline sometime soon!)

Two years passed before anyone put polysoup in without rewriting the rest of the engine to use some other physics SDK. Actually, I ended up being the one to integrate polysoup with DTS. :-/ I was surprised no one else beat me to it! I brute-forced it – every triangle as its own convex. Torque ran this approach like a champ as long as you didn’t feed it tiny triangles. Did the same trick again when it came time to write Atlas1 collision a few months later, but that wasn’t really “polysoup” in a meaningful way for anyone since it only imported heightfields.

It’s interesting to me that the current Torque maintainer (and all around good guy) Matt Fairfax wrote loaders for .3DS, Quake3, and Unreal formats, and the maintainers that came before me (Rick Overman, Mark Frohnmayer, and Tim Gift) have also done things of this nature (in fact, they are at least partially responsible for DIF and DTS). I guess you don’t get to hold the reigns of TGE unless you’ve written a couple 3d renderers/loaders.

I think in total I’ve done 4 loaders with renderers – Blitz3D, Chunked LOD (for Atlas 1), Atlas2, and OBJ. Really, OBJ doesn’t count. Every graphics programmer in the universe has hacked up a OBJ loader – from Carmack on down.

In conclusion? I guess there are a few morals to draw from this nostalgia-inspired post:

• Keep shots from your old projects. It’s fun to reminisce! I have a few hundred shots from Atlas development that might be fun to write about some day…
• Don’t be afraid to buck common knowledge and go out in a technical direction no one thinks is feasible. Try it small and if it works, keep going.
• Do projects for fun! Good stuff will always find an opportunity for reuse – be it some code, or a technique, or even a fresh perspective. The ideas and techniques I came up with for this project ended up helping me for years to come. Working with Blitz3D’s format helped me see Torque’s approaches in a fresh light.

Edit: Adrian Tysoe posted on GarageGames about his side of this experience – here’s what he had to say:

Heh, I remember working on this with you. I figured they were the starting point for some of the later torque advancements. Just wish that the DTS had been updated to support more than 1UV. Was pretty fun to work with and performance was decent.

The worst part was trying to match invisible torque terrain collisions to .b3d meshes just to get the nice water blending around the edges. The terrain used the coldet collision for the actual character and weapons etc.

Was pretty neat and I enjoyed having 2UV’s to playwith so I could bake my own lightmaps in 3dsmax of gile[s], in many ways the most useful thing to make TGE rendering look a bit nicer.

Here’s a couple more pics the project Jeremy and I started using it.

I spent some of my downtime over the past few months working on client-side brick physics for Blockland. The feature has finally been announced, so I can talk about it. The video shows off most of the features; the main one that isn’t shown is the interaction between players/vehicles and bricks; they will push bricks around but aren’t affected themselves. Watch it in full screen, it’s full 720p HD video.

There were two big problems related to doing physics for Blockland. The first was the scale of the problem – there can be well over 100,000 bricks in a single server, which is beyond the capabilities of most physics SDKs to simulate as discrete objects. I started with PhysX, and moved to Bullet after realizing that the PhysX runtime is 80mb, far more than could be included in the Blockland download (which weighs in at 20 mb). Both of these libraries broke in different ways with large numbers of objects.

At first, I implemented a management system for brick proxies, so that it kept them under the hard limit in the SDK (around 2**16). PhysX accepted this, but Bullet’s broadphase has some stuff that’s O(# objects) or worse, so it fell down. Eventually, I moved everything into the same system used for static world geometry, which was a grid of static meshes. It turns out that Bullet is a bit faster at creating these mesh objects than PhysX was.

The static mesh cache took quite a bit work to get solid. Because the simulation is for aesthetic purposes, it can tolerate a fair amount of “fudge,” which I take full advantage of. Nearly every kind of update is timesliced so that only a little bit is done each tick. This keeps things smooth, even at the cost of the physical state being inconsistent for a tick or two. Most updates are lazy, as well, only done if a dynamically moving brick, player, or vehicle comes into the area.

The other problem is the wide variability of Blockland user’s computers and usage patterns. Not every user has enough CPU to run physics. And every user has the potential to build something that is very resource intensive to simulate. I spent a lot of time implementing a “physics diaper” – logic to detect when physics calculations were taking too much time, and scaling back the simulation until it’s fast again. This takes two forms. First, if physics ticks are too slow, the simulation is decimated – every other brick on the list of moving bricks is converted to a lighter weight parametric simulation that doesn’t consider collisions. If they remain too slow, then eventually the physics simulation is disabled entirely, until the user turns it back on. This can help with very complex builds or very slow computers.

Thanks to good physics middleware (I include both PhysX and Bullet under the “good” category, even though PhysX is a little on the bloated side), I was able to solve a pretty tough problem – simulating motion for hundreds of thousands of bricks on commodity hardware – in short order. And I have to thank my friend Eric for making such an awesome sandbox and letting me play in it.

This gets kind of technical so I’ve hidden most of it behind the jump.

C++ And AS3 Compilation

So where am I? First step was to build the runtime, which was easy – compiles right out of the repo. Then I realized I couldn’t compile. That stalled me for a few months. Once we started working on the current project at work – which is in Flash 9 – it was a lot easier to figure out the compiler part. Flex 2 SDK for the win! It’s free, you see.

The relevant command to compile a .as file is:

java -jar SDK_PATH\lib\asc.jar myTest.as

Then you can run it with the binary you compiled earlier – just do:

avmplus_d.exe myTest.abc

(Paths are a little trickier than this in reality due to where the compiler sticks things, and if you did a release build the binary name might be different.)

Native Binding

The next thing I attempted was adding my own class. I’ll be cleaning up the code for this and posting it a little later – right now it’s sitting in AvmCore.cpp which is definitely not a good way to do it! But I will talk about the native-function-binding process, which is pretty slick.

Basically, there are two parts. First, some #defines that go in your C++ code and define some tables. They look like this:

BEGIN_NATIVE_MAP(TestClass)
   NATIVE_METHOD(TestClass_helloWorld, TestObject::helloWorld)
END_NATIVE_MAP()

BEGIN_NATIVE_CLASSES(AvmCore)
   NATIVE_CLASS(abcclass_TestClass, TestClass, TestObject)
END_NATIVE_CLASSES()

This matches up with some ActionScript code that looks like this:

package
{
   public final class TestClass extends Object
   {
      public native function helloWorld():void
   }
}

Now the weird thing here is that the C++ side references these magic files (see core/builtin.h and core/builtin.cpp for examples). builtin.h contains lines that look like this:

const int TestClass_helloWorld = 557;
const int abcclass_TestClass = 17;
extern const unsigned char builtin_abc_data[];

builtin.cpp is just a big array of bytes defining builtin_abc_data[] – AS3 bytecode, as it happens. Not too surprisingly, this code is one of the first thing the VM executes when it starts up. Where does it come from? What’s in it?

native keyword and ASC

There’s a file in core called builtin.as that references other .as files in the same directory, and in that is defined a bunch of stuff related to the core built-in classes like Function and Object and Boolean. Some of the methods are native and body-less – and sure enough there’s corresponding C++ code in the AVM – and others are populated with script code. (I assume that you have to have matching AS3 and C++ side functions or face all kinds of nasty bugs, but haven’t tested this yet.)

However – there’s no obvious way that the script code is mapped to C++ code except for these IDs defined in the builtin header file. Where do they come from? What tool made these files? They have all the marks of being machine generated – they’d have to be since there’s no way to manually specify a “native ID” for a method.

After some digging, I discovered the key to all this. It’s the AS3 compiler! If you compile a .as file with a native function in it, it automatically generates the C++ code to embed it in the AVM. That’s where all those weird constants come from.

Exception 0×80000001

So – I made a test class, regenerated the builtin scripts, and bound my own native functions. Neat! So I ran my test script to see if it would trigger the debugger (I set a breakpoint in my class implementation). And it did – but not in my code. Specifically, I got this weird exception:

First-chance exception at 0×00459272 in avmplus_d.exe: 0×80000001: Not Implemented.

Well, what the heck is that all about? It turns out that the JIT code generator, which produces machine code versions of your AS3 code, reserves a bunch of memory from the OS to build executable code in. On windows in particular, as part of the memory setup, it does it with a call to VirtualAlloc that looks like this:

void *res = VirtualAlloc(address,
o GCHeap::kNativePageSize,
o MEM_COMMIT,
o PAGE_GUARD | PAGE_READWRITE);

(This is in GCHeapWin.cpp line 317 or so.)

It turns out that PAGE_GUARD makes the first access to the specified page trigger an error. MSDN has more to say on the subject of guard pages. I haven’t fully traced this code yet but it appears that it’s part of a scheme to dynamically grow a buffer as the MSDN article implies. In any case, it’s not actually an error – just a side-effect of normal operation. This can be confirmed by running the AVM outside of a debugger – it functions correctly – and by continuing execution after the breakpoint is hit.

The details of how to suppress this error are different for every IDE, but in VC7, you go to Debug->Exceptions…, select Win32 Exceptions, then hit Add… – for number input 0×80000001, and description can be whatever you like. I’d recommend “page guard” or something like that. Then make sure that a breakpoint isn’t triggered by selecting it in the list and choosing “continue” from the options at the bottom.

Conclusion

Things we know how to do:

• Compile AS3 to ABC and update the builtins in the AVM.
• Compile the AVM from C++ source.
• Bind our own native C++ code to the AVM.
• Debug the AVM w/o extra exceptions causing problems under VC7. (The same steps should apply to VC8 as well.)

AVM is pretty cool. I look forward to working with it more.