Building and Deploying Heroengine

From HEWIKI
Jump to: navigation, search

Contents

He server.png Documentation on this page is intended for customers of HeroEngine managing their own server(s).

Overview

The build and deployment of a product with the scope and breadth of a MMO engine is complicated, spanning from the proper setup of a developers' workstations to build an enormous C++/C# project to updating large numbers of servers with new binaries and/or game content. Fortunately, HeroEngine includes both documentation and tools to simplify the process.

Before, we talk about the specifics of building and deploying HeroEngine lets take a look at the development environment that we use here to develop HeroEngine.


HeroEngine's Build/Deploy Environment

While this documentation mentions a number of third-party solutions, it is not intended to be a specific recommendation or an indication of a requirement that your company use them. Rather it is meant to describe our development environment as it suits our needs & requirements, and the products & infrastructure we have found to be useful.


HeroEngineBuildEnvironment.png

Although not required, the HeroEngine team uses JetBrain's TeamCity for backend builds and continuous integration. Third party build technologies including Xoreax' IncrediBuild may also be used, but are not required.

Once built, we utilize HeroEngine's (optional) Deploy system to package up all of the necessary files (executables, dlls, configuration files, etc) into an installation file (generated using Wise) for that product and places it on a Launch Server. Applications (such as HeroBlade or the fireupdaemon which spins up HeroEngine Processes) are started via a Launcher application that connects to the launch server to download and install a new version of the application if available.

We utilize Perforce as our Source Control Solution for all HeroEngine products.

For development and quality assurance purposes, we run a number of worlds at a variety of revision levels on multiple OSes. Utilizing HeroEngine's World Push and Live Update technologies we are able to update content (scripts, DOM, GOM and art resources) between worlds as needed.

Building HeroEngine

Developer Workstation

Proper setup of a developers workstation is a critical first step in getting ready to build HeroEngine and is a fairly lengthy process which includes the installation of:

Workstation Setup

Our developer workstations are multi-core computers with plenty of RAM, a spacious hard drive (mirrored or not), a modern discrete graphics card and dual-monitors. Our philosophy is that developer time is expensive and things that can speed up their velocity and/or improve their workspace tend to pay dividends in completing milestones on time.

Minimum Hardware recommendations

Build Server

The build server is NOT a required element for building HeroEngine, its still probably a good idea though...

There are a variety of reasons why many developer teams consider the use of a build server to be essential to the development process:


The HeroEngine team utilizes a combined continuous integration / build / deploy / symbol server that is build on a 8-core 8-gig machine running 64-bit windows 2008 or later OS. The setup of a build server is roughly the same as setup for a developer workstation additional complexity beyond that of a workstation is introduced by the build system(s) chosen by your team. Our setup allows us to build a product like the Deploy Client or Deploy Server in seconds on a single core or Heroblade in approximately six minutes (most of that time is spent in disk I/O and memory/CPU consumption building *.dll and *pdb files).

IncrediBuild

IncrediBuild is NOT a required element of HeroEngine, but you may find it useful in your source code development.

We have had a very good experience using IncrediBuild by Xoreax Software (http://www.xoreax.com/) to speed up the build process.

Source Control

At a minimum, you will need the Perforce Client Software (a free download from Perforce.com) to access the source code branch for your company in our Perforce depot. Using the Perforce Client you will be able to sync to the branch for your company located in our Perforce depot allowing you to access the latest release of HeroEngine.

Beyond that, for reasons that we hope are well understood your development should be under source control (whatever version of it that you prefer). Smaller companies without IT support staff or security needs can probably consider using git, mercurial or other new DVCS's (Distributed Version Control (Systems)). Ultimately, if your company needs to keep things private (which eventually is every company) there is a need to use a version control system that has security & authorization built-in and not just added on as an afterthought.

Depending on the source control solution you choose, you may or may not need a centralized server.


We have found Perforce to generally suit of our needs.


Building the Solutions

Users of Vista/Windows 7 with UAC enabled MUST run devenv and/or run the Visual Studio Command Prompt with administrator privileges OR turn off UAC to properly build.

Preconditions


The most commonly built solutions during development are:


While you can build the solutions in Visual Studio there are some dependencies between projects (for example the firestorm libraries are used both in server and client applications), it is more common to utilize a utility script like Rebuild stuff or located in the root of every branch as it produces a repeatable build process. There is also a sample command (simu_release.cmd) located in the root of the branch which illustrates the proper order in which the various products should be built to create a "release".

If you are building your branch for the first time, a good place to start is by using rebuild stuff to rebuild HeroEngine's "core" solutions.

rebuild_stuff dp hj hb pc rb pub wp mcc /nop4


Building the solutions will generally be all that is necessary, but some applications expect the existence of configuration files which are normally generated and installed via the deploy process. Since we are working directly with the source rather than an installed product some applications will require that we create an appropriate configuration file in the Run directory for the solution. For example, HeroBlade expects a HeroBlade.cfg to be located in the run directory the easiest way to determine what the contents of that file should be is to copy the file installed by your game specific HeroBlade.



To locate your existing HeroBlade.cfg file:

  • Click on the Start Menu
  • Navigate to All Programs\HeroEngine Tools\
  • Right-Click on HeroBlade<CompanyPostFix> and select the Properties from the Menu
  • Click on the Open File Location button

HeroBladeHEDevWindowsShortcutProperties.png

  • Using the explorer window, copy the HeroBlade.cfg file to the <BRANCH NAME>\HeroEngine\MAIN\heroblade\Run directory


Deploying HeroEngine

DeploySystemOverview.png

Deployment covers several levels of distributing new functionality/content to your servers and users. HeroEngine includes several tools intended to assist with the deployment process:

We're going to start off by discussing how the binaries you have just built get distributed to your server cluster in a manner that is appropriate to the "version" that a world ("shard") is running.

Deploy System

The deploy system is NOT a required element for building/deploying HeroEngine, however you will need the moral equivalent of it even if that is "copy executables, generate config-files, and update installation values by hand"...

The deploy system provides HeroEngine with an easy to use centralized tool for the automation of the deployment of HeroEngine product updates supporting the deployment of executable product updates such as HeroBlade or the Servers. It allows the developer to create specifications using the Deploy client for products that list all of the necessary files to include in a installation of a product such as HeroBlade. Once a specification has been created, the deploy client can be used to initiate a product deployment resulting in the creation of a installation executable for the product (currently utilizing a third-party technology called Wise Installation Studio to generate the installation files).

Once generated, installation files and the corresponding configuration files contained within are stored on the Deploy Server maintaining a full version history of all deployments and then copied to the Launch Server which will distribute the installations when applications are launched via the launcher application (by default, all HeroEngine applications both server and client based are launched via the launcher application).


Deploy Server

DeployServerExecutableCreation.png

A deploy server is responsible for distributing a deployed application to the Launch Server and maintenance of a repository of previous deployments (i.e. their installation files). The HeroEngine team utilizes a combined continuous integration / build / deploy / symbol server in the form of a 8-core 8-gig RAM machine running Windows 2008 Server 64-bit or later OS.

Prerequisites
How the Deploy Server Works...

When a product is deployed, the deploy client copies files from the source directories on the machine where the client is run based on the deploy specification for the product (in our case, this is done on the Build Server) to a directory on the Deploy Server named deploy_incoming and from there it is copied to directory based on the specification appropriately tagged with a new version number.

The deploy server then creates a copy of the base.wse (wise installation script, a copy of which you can find located in <BRANCH NAME>\HeroEngine\Main\tools\deployment\base\base.wse in your perforce branch) and makes modifications to it based on the product deploy specification including information about where the files are located. The modified wise installation script is then passed to the Wise.exe along with the product deploy specifications resulting in the creation of an executable installation file. The installation file is then copied to the Launch Server and the Launch Server's configuration file is modified to identify the new current version for the product.

Additionally, debug symbols are copied to a directory on the Deploy Server.

Versioning
DeployServerInstallImages.png

The deploy server functions as a versioned repository of every deploy ever made allowing you to easily revert to previous versions should a bad deploy be released.


Deploy Client

Launch Server

So What If My Company Does Not Want to Use the Launch/Deploy Systems?

If you choose not to utilize HeroEngine's Launch/Deploy Server mechanics, then you need to ultimately duplicate the functionality of distributing and applying updates. This could be done by hand, or via an existing mechanism developed by your company.

The main difficulties lie in that you need to:


Required Files by Product

The following pages contain images of the Deploy Settings and Wise Compiler Parameters used by the HeroEngine team to generate the installation files for their respective products.

Please note, the compiler parameters are provided as examples but the parameters you would use are customized to your product and consequently WILL be different in many cases.

Updating Content/HeroScript Code Between Environments

PushToolsHypotheticalDevelopmentEnvironment.gif

Sooner or later, you will expand your development environment to include more than a single world. Typical configurations include worlds for experimental development, testing integrations, quality assurance, and ultimately we all hope lots of production servers for your users. Source code development is (commonly) distributed via the deploy and launch server mechanics.

That leaves unanswered the question of how do you move content between worlds?

First, lets define the content that needs to be moved:

HeroEngine supports two different methods of moving these types of content between worlds:


While the image depicts a hypothetical environment where two developers are working in isolation and then pulling content into an integration world, this is in fact atypical of how the engine is used. After all, one of the big features of HeroEngine is the unique ability to develop a MMO collaboratively, live in real-time. More commonly, you have a development world in which the majority of development is done (you might call this the "mainline") and based on need you may spin up additional worlds to develop experimental or large scale changes that require isolation from your mainline to minimize the impact on the rest of your team.


World Push

World Push

Typically, MMOs distribute new content via pak files that include all of the new game data and content. In a development environment, that means that game logic and new content can not actually be used or tested until the next build cycle is complete and the pak files updated. For the end-user, pak files are typically distributed on the dreaded "patch day". While supporting the concept of patches, HeroEngine supports a much more organic workflow for game designers and world builders where their modifications take affect in real-time. Leaving the question, how do you update other worlds with new data in HeroEngine?

A typical HeroEngine deployment will often include layers of world servers that are used to isolate one or more development and QA worlds from the production game worlds. World Push serves as one of two potential mechanisms, the other being Live Update, by which game data and content may be moved from one world server to another.

The amazing thing about this process is that it does not require excessive downtime as it can be done in as few as a couple minutes! That means you can update your QA worlds, public test or even production servers with only a few minutes of downtime.

Even more amazing is that most of this process can be done with no downtime at all! World Push's companion, Live Update, is capable of doing almost everything World Push does while the servers are still up and running!


A command line version of the world push application is available support automation of world updates.


World Push is available as a download from the installations page generally found on the apache webserver running on your development database server.

Live Update

GOM Differences Selection

Live Update allows you to pull game content (scripts, Assets, DOM definitions, data files and areas) to live, running Worlds without having to bring them completely down.

In other MMO server architectures, game content updates require all game servers to be brought down before they are updated with new data. This blocks developers and players from interacting with the game/development worlds for the time that the servers are down. Live Update avoids these cumbersome and wasteful steps by only requiring downtime for areas that are being updated. Game content can be pushed on-the-fly to any running game server.

Live Update is exposed via panel in HeroBlade.


See Also

Personal tools
Namespaces
Variants
Actions
Navigation
Toolbox