XDR R&D #2: The Framework

Introduction

In this blog post, I will be covering the base framework we collectively constructed within PhyreEngine to start as a basis for our PS4 racing game. I’ll be covering the motivations behind some of the decisions when constructing the framework, the structure of the framework itself and how it works, as well as any loose ends/unknowns/risks that the structure carries. I will finish off with some thoughts about how it could be better approached if I did it again, as well as potential future improvements.

Motivation behind the framework

When we worked on our iOS game Godsend using cocos2d-x, we encountered loads and loads of issues, and time was spent where it shouldn’t have been. Thinking back, I really should have covered these issues in my previous blog posts about the iOS title, but most of them were left undiscussed, I suppose mainly because it would have been too expensive (time-wise) to resolve them. Therefore, I will try to briefly discuss these issues as they served as the basis for how the framework was constructed for this project. The main goal for the framework was to save time for the programming team in as many aspects as possible, so I will be looking at where time was wasted in the iOS game and how we can learn from that and construct a framework that mitigates those issues.

First of all, inheritance based hierarchies seemed to cause a lot of trouble for us in the long-run (which is what we used for cocos2d-x). This is really easy to illustrate with an example from our iOS game. When we started off the project, we knew that we would need to implement moving platforms. And so we implemented them, inheriting from the base class CGCObjPlatform. Later on, we needed to implement background sprites that move back and forth. This is exactly the game behaviour that a moving platform has. However, background sprites are not a platform, they do not have a collision box attached to them, they are not created through the level editor, and the list of differences goes on. Furthermore, while CGCObjMovingPlatform inherits from CGCObjPlatform, CGCObjMovingBackground would inherit from CGCObjBackground. But to re-use the functionality of the moving platform in an inheritance-based framework, we would have to abstract out a class such as CGCObjMovingObject, then have these two classes inherit from it. But what does CGCObjMovingObject inherit from? This is just one trivial example (potentially blown out of proportion), but that’s the kind of problems you start running into when your project grows bigger and bigger, and there’s more and more different types of objects in your game.

One solution to that problem would be to use composition instead of inheritance, and this is the main driving force of our framework for the PS4 game. Essentially, this is how a lot of modern game engines work, such as Unity: entity component based systems, also known as data-driven programming. When looking into component based systems and how to implement one over Christmas break, this article really helped me out: http://www.richardlord.net/blog/what-is-an-entity-framework . It discusses much of what I’m discussing right now – shortcomings of inheritance based frameworks, and essentially explains what an entity system component framework is.

Such a framework solves the issue mentioned earlier in terms of moving platforms and backgrounds. We could just make a MovingAtoB component, and attach it to both types of objects. Furthermore, when I spoke to the visiting programmers from Playground Games, they strongly agreed about the strength of this approach, also saying that when enough generic components are written, creating new functionality becomes a lot faster, since components are highly re-usable (if written correctly) and the task breaks down to picking the existing components you need, and then maybe writing a few more to implement your object.

To put it bluntly, I honestly thought that if we were doing our iOS game in Unity, we would have made it in half the time we took making it with cocos2d-x. However, untangling inheritance chains is surely not the only reason why coding in Unity is so much faster, so I had to consider other advantages that Unity holds over our iOS framework.

The second advantage is the amount of time spent connecting our game to the OGMO editor, the amount of time wasted changing minor details as requested from the design team and the amount of time spent modifying/extending functionality manually, the amount of time spent importing artists’ work, helping them check how it looks in the game and so on. If I had to put it in a diagram, the project went a bit like this:

iOSTeam

Where for almost all issues, the art and design teams would need to go to the code team, and have barely any access to the game themselves. If an artist drew a new object for our game, they have to go to the code team to attach those sprites to some sort of object and hook it all up. Same goes for the design team – any changes in requirements, balancing and so on would usually involve code work.

From a code team perspective, this is massively inefficient, especially considering that I, for one, had to manually insert all objects into the level editor before they became usable by our level designer. A lot of the time this wasn’t even real programming work, but data entry and tweaking. So instead, we figured that for the PS4 project, we would like to achieve something like this:

PS4Team

So that the art team can test their models/etc by putting them into the game themselves, the design team can tweak/enter data themselves and manipulate game logic to the extent they need to by combining building blocks and flicking various switches.

I think the key observation here is that games are mostly data-driven. The code team should be writing controllers that manipulate the data, i.e. the game logic, while the art and design teams should be supplying models, stats, damage/speed/whatever values – all of the data that the logic operates on and the end outcome hopefully looks somewhat like a game.

By eliminating this barrier between artists/designers and the game which is the code team, everyone in the team should be able to work a lot faster and more efficiently. The artists have a quicker turnaround in terms of inserting their work into the game, since they can do that themselves. The designers can do balancing/tweaking quicker since they are directly in charge of the data that the game needs. And most importantly (to us programmers), the code team can focus on implementing the game logic required.

The interface that allows to achieve all of this is the level editor. I think this is one of the strengths of Unity – the development team can simply drag and drop game logic (scripts) onto game objects to make artwork do something useful, tweak values and connect logic to build levels/balance gameplay and so on. Fortunately, the level editor in PhyreEngine is quite strong in that regard, as it’s mainly data-driven and seems to expose enough data to enable this kind of workflow (at least way more than the OGMO editor did, as it only established a one-way communication link from the editor to the actual game).

I think it’s worthy to note that even when an engine provides this kind of interface for the designers and artists, it’s really easy to make it all useless by the code team. Simply put, the level editor needs to be the central pillar of all implementation tasks, and all functionality that’s being implemented needs to be geared towards use in the level editor. There is a really good example of this pitfall in the PhyreEngine samples themselves – the SpaceStation demo (which, admittedly, is quite old by now and was built when PhyreEngine was probably way more bare-bones than it is now). In it, all game logic is coded via classes (not components), and it certainly does not offer much for the designers/artists to play with in the level editor, other than placing assets around to create the level in the broadest sense (bunch of environment assets that the player can traverse). I think we can learn from that example and focus better on exposing as much of the game logic to the editor as possible instead.

Finally, the last bunch of motivations that do not need much explanation are listed. One of the goals outlined to us was that the artists should be able to access a “quick build” of the game, i.e. drop a new vehicle model that they’ve just made, press play and drive it around to see how it feels. With a component based framework, they can do just that by importing the model, attaching the component(s) necessary for a vehicle, and pressing play. The final motivation (and this might be a stretch goal, seeing as Phyre does not have any UI system) is to enable artists to build menus in the level editor themselves. In the iOS game, the menus were all hard-coded, but they needed to change quite often, which is definitely inconvenient, as a code team member would need to get involved when really it’s all just changing the placement of sprites and such.

Structure of the framework

Now that I’ve gone through the long list of motivations for creating a data-driven framework, I can finally discuss what I’ve implemented (which should hopefully be more brief, as it’s kind of the answer to the host of problems that we’ve had with our iOS game). I’ll start off with a diagram of the framework (which is not 150% up-to-date, but it’s close enough to illustrate the idea, and the changes made are quite minor anyway):

FrameworkDiagram2

(Note: a full-sized version of the diagram can be found at https://drive.google.com/open?id=0B_BnvnFZLH7mWThJeGY3cnBlUms )

This is our best attempt at an entity-component based framework. Our code lives between two different projects, PhyreDeveloperExtensions and GCRacingGame. I’ll explain why that is a bit later on, but here’s how the solution looks in visual studio (for the most part, it’s ThirdPersonGameTemplate copied over and trimmed down a bit for our uses):

GCRacingGameSolution.PNG

(I will not cover the Scene classes, since Radu implemented the scene system and I’m sure he’ll be covering it in detail in his blog)

Thankfully, PhyreEngine is built using entities and components already, making this slightly easier for us. However, the component system is mainly geared towards Lua scripting (which makes sense – it’s probably a more suitable language for high-level gameplay scripting than C++). Knowing that, we needed to adapt this system towards C++ and we found the support for this in the PhyreEngine Custom Components.

PhyreEngine allows coders to write their own components in C++, which can then be attached to objects via the level editor. That’s good enough as a starting point.

Unlike Unity scripts, these components do not support game logic by default. By this I mean that they do not have an Update function. So, for starters, I’ve created a very basic custom component called CGCComponent, which all of our components are supposed to inherit from. This has an initialise, update and shutdown function, but we need to explicitly call these functions somehow.

The main class in any PhyreEngine game is a class inheriting from PApplication, which, in our case, is the CGCApplication. This has all of the functions for initialising the game, updating the game and shutting down the game. After a bit of digging around, I’ve found that we can actually find all CGCComponents in a given level and access them in the CGCApplication. This means that when the application is initialised, we can gather all of our components and initialise them, same for updating and shutdown. This gives us a component system and a basic place to put our gameplay code in (which was a major question for the code team – “Where do we put our gameplay code in Phyre?”).

So, all that the coder needs to do in order to create their own components is to inherit from CGCComponent and override the functions. An example of this is the CGCObjScoreTracker in the diagram (the component basically does nothing at all and was created purely to demonstrate how one would use the framework).

The first thing to note is that all components need to live in the PhyreDeveloperExtensions project. Otherwise, the PhyreAssetProcessor will not be able to detect them and will not know what to do with them. This creates the necessity of working in two Visual Studio projects simultaneously. What’s worse, PhyreDeveloperExtensions is a library project and is not directly executed, which brings a whole new bunch of problems to the table (some of which I’ll cover in this post, some in the other ones).

One of the major headaches with Phyre was that you need to explicitly bind the classes with Phyres object model. Otherwise, Phyre will not have a clue about how to interpret these components. This is done in CGCClassBinder. The Phyre User Guide tells us to do this in PhyreUtilityDeveloperExtensions, but I wrote a wrapper class so that we don’t have to heavily modify the Phyre classes present. I initially tried to automate this process altogether by writing macros that, when included in the header/cpp of our custom components, would automatically bind and instance themselves. These macros defined Bind and Instance functions and a static struct that would register these functions with CGCClassBinder when instantiated. The CGCClassBinder would then only need to loop through the registered functions and call them all. An outline of the macros and the previous implementation of CGCClassBinder is shown below:

FrameworkDiagramMacros

FrameworkDiagramClassBinder

However, this implementation worked up until I found out that our custom components have to be in PhyreDeveloperExtensions, and not in our GCRacingGame project. This means that the static variables would not be instantiated unless the class is explicitly used in the startup project somewhere – which beats the point of doing this at all, since the coder would still manually need to include their components somewhere every time. So instead, we’re stuck with explicitly binding and instancing our components. To reduce the code duplication for all of them, I’ve at least implemented a macro that does the common parts of instancing components.

What’s worse, every time you create a custom component, you need to (re?)build the PhyreAssetProcessor so that Phyre knows about your components. This is a separate Visual Studio solution and opening it every time to rebuild is a pain. I figured I would do as much as I can to make this easier, so I looked into rebuilding solutions from the command line and wrote a bunch of handy scripts:

RebuildAssetProcessor.PNG

These are included in our template solution, hopefully making this necessary step a bit less cumbersome.

That pretty much covers the base framework structure. The nicest thing about it is that we now have a way to attach gameplay logic dynamically through the level editor, instead of hard-coding the bindings between very specific game objects/models and their gameplay logic.

Furthermore, we can easily expose variables to the level editor by including them between the PHYRE_BIND_START and PHYRE_BIND_END macros, so that enables game designers to tweak gameplay.

Finally, components might need to interact with each other in one way or another (well actually, rather than with each other, some components might use others and so on). A dull example is a VehicleComponent which governs how a vehicle is driven. Both the player and the AI might use this. So the coder can create a PlayerInput and a AIInput component, which would pass on inputs to the VehicleComponent, allowing them both to re-use the exact same mechanism. I’ve actually implemented a similar system in Unity before for Dungeons of Rogueness (http://mindaugaskadzys.com/dungeons-of-rogue), where the AI and the player would both move in the exact same fashion, so I’ve created a MovementScript that handles it, and the player/AI controllers just pass inputs to it.

It turns out that it’s quite easy to access other components from a custom component. The current implementation of CGCComponent fetches the Node that the entity belongs to for user convenience, as well as the rigid body on initialisation. Therefore, the VOnInitialise() implementation of the class serves as a good example of how to do this kind of thing:

GettingComponents.PNG

So, on a high level, it seems that our framework supports building a game via an entity-component based system. I can now go on to discussing the shortcomings of the framework, as well as potential future improvements.

Shortcomings of the framework and Future improvements. What would I do differently?

When we got our hands on Phyre, I was already kind of biased towards having a data-driven entity-component based system for all of the reasons I’ve stated above. This might have made me slightly oblivious to other approaches that could be taken when using PhyreEngine. Furthermore, there were a lot of hurdles I had to go through to implement this and a lot of workarounds were needed.

The biggest shortcoming of the framework is the fact that we have to put our gameplay code in a library project that’s separate from our main project. This adds a layer of confusion to the framework as well as additional hassle. There is quite a lot of markup required for custom components in the way of macros, there’s the need to add code to a different class in order to bind and instance the components and so on. And then there’s the need to rebuild the asset processor after you add a new component.

This makes it quite easy to forget one of the necessary steps, then sit in front of the screen baffled as for some reason it doesn’t work. To help mitigate that, Radu wrote a guide on creating custom components (http://gamercamp2015.pbworks.com/w/page/104615857/Custom%20Components) and I wrote a skeleton class for creating new components (http://gamercamp2015.pbworks.com/w/page/104615587/Skeleton%20code%20for%20new%20Custom%20Components).

In my opinion, the benefits of components outweigh the initially steep learning curve of getting started with them. I would easily forget to do some of the steps at first, but the more I used these components, the easier it became. My only fear is that use of components within the team is still quite limited, and not all team members might be convinced about this approach/may not fully understand it. I think I should have done a better job when we were discussing the framework at explaining all of this, and gave a better intro to this structure to the team. I’m hoping to improve upon this in the next module.

Furthermore, I reckon it might have been possible to modify the asset processor to also process components from our GCRacingGame project. It’s questionable how much time it would take to attempt this, but it would solve the problem of working across separate projects and would allow for automatic binding/instancing of new components. Also, PhyreDeveloperExtensions is included in every Samples solution, which kind of means that all samples would contain our game-specific component classes. If I did this again, I probably would look into including GCRacingGame into PhyreAssetProcessor.

I would consider creating data-only components (which is possible in the PhyreLevelEditor), which would enable designers to manipulate data, and create classes in GCRacingGame that are instantiated for every type of data-component, and then manipulate this data. The reason why I did not attempt this before was that accessing other components would become complicated, as there’s no direct link:

DataComponents

So each entity (game object) might have a player controller and score controller, but how does one access the other? I think if I spent more time considering this, I might be able to create a way to link these up. For example, at creation I could maybe create a proxy entity object that would just be the container of all controllers, and then let each controller know about this proxy entity. This could allow controllers to query the entity for other types of controllers.

EntityProxy

This kind of implementation would probably get rid of the following problems: working across two projects, having to rebuild the asset processor, having to write so much markup for each new component. If anything, I will try to investigate this approach in my spare time next module.

If anything, the main problem with the existing system is the ease of use for coders and the amount of steps required to use it, so I will continue to investigate ways to simplify it.

I think I can also eliminate the explicit component Binding/Instancing step by writing a short Python (or any language) script that scans the Components folder and adds all discovered classes to a text file. I would attach this script to the build of PhyreDeveloperExtensions so that it adds any newly created components to the text file when you build, and then CGCClassBinder would scan through the file and Bind/Instance all of the components. This would sort of automate the need to touch CGCClassBinder, as well as any of our custom macros in each of the components, but I will need to investigate the feasibility of doing this.

Finally, I should look into code-generation in visual studio to make creating a skeleton for a component easier. I have no experience in this, but a brief google suggests that Visual Assist should potentially be able to do something like this.

Related files

None due to NDA.