Dynamic World Server Load Balancing and Allocation
HeroEngine has supported a sophisticated configuration mechanism allowing fine grained manipulation of the configuration of a "cluster" and its resources to a world or worlds. HeroEngine's Dynamic World Server Load Balancing and Allocation feature expands the capabilities of the original configuration mechanisms to leverage cloud computing resources. A world spins up on the best available server, as determined by a variety of algorithms; by default, using a server max headroom algorithm.
- Maximizes resource availability for each world
- Worlds are given a relative "weight" (the comparable use of resources) which is then used by the new algorithms
- Increases Data Center and Cloud Computing resource flexibility
- Simplifies world configuration
- Static Processes are no longer confined to a particular physical server
- Adding/Removing servers no longer requires manual reconfiguration of individual world configurations
- SLEEPING - World is currently SLEEPING and inaccessible until it is woken up
- WAKING UP - World has been scheduled to be woken up and will transition to SPINNING UP in the next few minutes
- SPINNING UP - World is currently in the process of spinning up and will be UP shortly
- UP - World is currently running and the user's client matches the version of the world. The world is accessible.
- SPINNING DOWN - World is currently transitioning from UP to SLEEPING and can be reawakened once it is SLEEPING
- SHUTDOWN - World is intentionally shutdown and can not be woken up automatically
- DOWN - World is unexpectedly down and can not be woken up automatically. HeroCloud customers should contact support if this is not the result of scheduled maintenance
- OUT - World is UP, but is not available to user's client for login as the service directory has no transports allowing the client to talk to the server
- OLDER - World is available for login, but the user's client version number is newer than the server's. Connection to the world is possible with the current client, but depending on the client changes it may corrupt data. It is recommended you not use newer client than the server requires without intimate knowledge of all of the changes involved
- NEWER - World is available for login, but the user's client is older (out of date). Connection to the world will not be possible with the current client.
How does this affect...
...My HeroCloud World?
The first thing you will notice is that a new state (SLEEPING) appears in the Service Directory Listing for your world. When a HeroCloud world has not been in use for a certain period of time, it enters a suspended mode called SLEEPING. The SLEEPING world can be woken up by authorized users when they attempt to log into the sleeping world using HeroBlade or the Repository Browser. The process of waking up a world can take a few minutes depending on a number of variables including whether or not there are other pending spin-ups or additional capacity is required.
...My Unmanaged World?
Unmanaged worlds (i.e. worlds where the customer is handling the servers themselves) can be configured to utilize or not utilize the Dynamic World Server Load Balancing and Allocation capabilities. It is typical in a development configuration for an unmanaged world deployed on a minimal set of physical servers to choose not to utilize these capabilities for a variety of reasons including; ease of use, predictable process location, debugging requirements, lack of hardware or appropriateness of the benefits to their unique organizational needs.
See Advanced Usage below for configuring an unmanaged world to stay up.
Awakening a World
Awakening a SLEEPING World is quite simple, requiring the World be selected and the "Awaken World" button be clicked. Once a world is scheduled to be awakened, no other action is required until the world has completed its spin up. The process of awakening a world can take several minutes depending on a number of factors including number of pending world spinups and whether or not additional capacity will be required. The status will change to SPINNING UP once the cluster has started to spin the world up and will finally transition to UP when the world is available for login.
Putting a World to SLEEP
Worlds that are not configured to be kept up (see advanced usage below) will automatically enter the SLEEPING state after a period of inactivity. There is no special activity required for this to occur.
The Dynamic World Server Load Balancing and Allocation features are composed of both significant modifications to the database schemas as well as the Master Control Daemon process enabling support for dynamic (automatic) reconfiguration of worlds to utilize available resources as defined in the database. There are a number of new concepts expressed in the database including:
- Server Definition - defines a physical resource and its characteristics. Server Definitions have a security policy that controls how ports are allocated to HeroEngine Processes. Server definitions are be "closed" to prevent worlds from being reconfigured to spin up on them.
- Security Policy - defines a set of parameters used during reconfiguration to properly configure the various ports utilized by HeroEngine's processes defining among other things; poRouterPort, DudeManagerTransport(repository and service directory), dynamic port range and slot range
- Slots - Slots are used in conjunction with the security policy for a server to determine the specific port allocations for a specific world dynamically configured to run on that server. Slots are unique per server
- Spinup Group - Spinup groups define a set of server definitions upon which a universe may be dynamically reconfigured to run
Disable Inactive World SLEEPING Transition
Worlds can be configured to not automatically enter the SLEEPING state by setting the parameter KEEPUP_ENABLED to the value of "1", which can be done either via Master Control Console's configuration editor or by right-clicking on the universe in Master Control's World View and selecting "Keep Up".
Modify the Inactivity Spindown Delay
By default, the inactivity spindown delay is set to thirty minutes. This value can be modified to a minimum of five minutes by configuring the parameter SPINDOWN_IDLE_DELAY at either the default or universe level using master control's configuration editor.
Enable Auto-Spinup for a Universe
Universes can be configured to automatically spin up when master control daemon is run. Auto-spinup mechanics assume that the configured server (fireup daemon) for statically configured universes is running (whereas universes that leverage reconfiguration mechanics just need a server in their spinup group to be running). To enable auto-spinup for a universe two configuration values (AUTO_SPINUP and AUTO_SPINUP_ORDER) must be set using master control's configuration editor. Universe services such as a shared service directory are typically configured to spin up first (i.e. use 0 or a low number so it spins up first).