An 'ID' is an unsigned 64 bit integer used to identify GOM and DOM data in the database both locally and on the server. Each GOM node or DOM definition is linked to a unique 'ID' and will never be re-used (even if it is deleted).
'ID's are used by the engine to uniquely identify persisted and non-persisted instances as well as to identify class and field definitions within the DOM. These 'ID's are managed by an ID Server, which is responsible for distributing blocks of IDs to Area Server instances so that they may facilitate edit requests for the creation and manipulation of object instances. Each object instance corresponds to one and only one 'ID': a contract which the ID Server enforces by maintaining sole responsibility for distributing new blocks of IDs to area servers. IDs may also be acquired and assigned via HSL script, either explicitly through the use of HSL ID Spaces (which come from separate pools of IDs) or implicitly through the creation of persisted or non-persisted objects. IDs are also used heavily in the CLI to effect changes to DOM definitions. Finally, since the space from which IDs are assigned has a size of approximately 2^64, the risk of a world exhausting all possible IDs is very small.
What problem(s) does this solve?
- Providing a means of uniquely identifying all game objects and data definitions within a world
- Providing a large (~2^64) space for ID creation supports extremely large worlds
- Provides a streamlined pipeline for acquisition of IDs which remain global across all areas in the world
- Provides multiple interfaces (CLI, HSL and HeroBlade edit commands) for creating or modifying GOM and DOM data.
What problem(s) does this not solve?
- Potential conflicts in ID spaces between worlds inheriting from the same base data
The relationship between IDs, areas and the ID server is illustrated in Figure 1. The ID Server - being the sole producer of IDs - is responsible for fulfilling requests by area servers who are in need of IDs. It does this by reserving ranges of IDs from the database and distributing these IDs to area servers in need. The area server - once supplied - uses the provided IDs to facilitate creation of persistent and non-persistent nodes as well as DOM definitions.
Each world has an ID Server which is responsible both for managing the set of IDs currently in use in the world as well as reserving and distributing new IDs to Area Servers as they make requests for them. Upon receiving such a request, the ID server will reserve a block of IDs of some fixed (configurable) size and deliver it to the requesting area server. These IDs will be pulled from the ID line associated with the current world and their values will increment starting from the value of the end of the previous block and spanning the range indicated by the currently configured block size. This process is illustrated in Figure 2:
In either case, the area server will fire off a request to the ID Server for a new block of IDs. Upon fulfillment of these requests, the area server will have access to an additional block of IDs with which to assign nodes.
Note: It is entirely possible for an Area Server to exhaust all of its IDs before the ID Server can deliver (asynchronously) a new batch. In a normally functioning world, this is an extremely unlikely event, but an out-of-control script or incredibly dense area import can cause issues. Solutions to such an incident, should it occur, include performing smaller asynchronous operations in-script and increasing the block size for allocated IDs.
When an area server spins up and receives its block of IDs, it is then capable of handling requests from the editor (or script) to create new nodes and DOM definitions. These entities are allocated IDs incrementally starting with the least-most available in the area's list of assigned IDs. As entities are created and IDs are consumed, the list of available IDs will shrink until - at some point - a new batch is requested from the ID Server. This then refreshes the local store of IDs and the process may continue.
An area that shuts down prior to using all of its assigned IDs will drop all of its remaining IDs and they will be lost. There is no aggregation and recycling of unused IDs, as this would be extremely inefficient to track and manage. For large multi-world universes where collisions become more likely, other considerations need to be taken into account (see Partitioning below).
In multi-world setups (e.g. development world, QA world, multiple production worlds, etc), collisions of IDs must be strictly avoided. If changes are made in one world that begin consuming IDs that were previously assigned to the production world, the ID spaces begin to overlap and world data becomes corrupted. To avoid this, we can partition the ID space such that no reasonably-sized world will consume more IDs than are contained within its designated partition. This ensures that any push of data from one world to another does not create an irrecoverable corruption by overwriting or otherwise impinging upon the ID space of the destination world.
Partitioning need take only the following into account:
- The size of the total ID space for a single world is 2^64-1
- HeroEngine reserves the rights to IDs 1 to 2^63-1
- This leaves IDs 2^63 to 2^64-1 up to the consumer
This is illustrated in Figure 3:
In order to guarantee no collisions between ID spaces, we need to make the dev ID space (or - alternatively - the ID space of any world from which you might someday wish to initiate a push of data) large enough to contain any possible 'reasonably sized world'.
Determining the total ID space requirements for a development world is left as an exercise for the customer and involves the following criteria:
- frequency of ID allocation for objects created by server scripts
- size of ID batches
- batch minimum percent (the consumption threshold at which new IDs are requested from the ID server)
- frequency of area spin-up
Again, and this bears repeating:
- It is critical that production servers not allocate IDs in a range overlapping the development server(s) if the ID Line is not shared. Failure to address this will result in an inability to publish development content to the production worlds.
The following table describes the manner in which the ID Space is partitioned by the HeroEngine team.
|HeroEngine||Legacy Definitions, Areas|
|HeroEngine||Client-side temporary IDs|
|HeroEngine||Legacy Definitions, Areas|
|HeroEngine||HeroEngine Base Clean Packages|
|HeroEngine||HeroEngine Full Customers|
|HeroCloud||HeroCloud Development Worlds|
|HeroCloud||HeroCloud Staging Worlds|
|HeroCloud||HeroCloud Production Worlds|
The following quantities may be configured for IDs:
- ID Line Starting Value
- ID Block size
- Area Server ID consumption threshold
IDs make creating, accessing and modifying data very straightforward. They can be used explicitly to reference a node or definition via HSL or CLI commands, and they are also embedded into visualized nodes in HeroBlade.
To illustrate the major methods of using IDs, let's go through a few simple examples:
Example 1: IDs in HeroBlade
Let's say we have an area that we've been building for quite some time now. It contains dozens of asset instances, a couple of NPCs and various regions, triggers and pathfinding nodes. Now - selecting any of these assets (let's just say a Tree) and bringing up the Properties panel will reveal a field named GUID. This is the unique ID assigned to the selected instance and may be used as a reference for use in HSL scripts or CLI commands. Whenever a command is issued with one of the built-in transformation widgets or an asset instance is added via the library, this ID is used internally to act as a reference for the node in question. Each node visualized in the HeroBlade client is associated with one of these Global Unique IDs.
In this case, our tree instance has an ID of 131959292244.
Example 2: IDs in the CLI
Now, let's take our tree that we selected in the previous example. Let's say we want to modify its properties from the CLI (perhaps we want this to be a tree that has additional functionality built into it that we can't visually inspect or edit with HeroBlade). In order to do so, we first need to know the ID of the instance; this is something we discovered in the first example.
Now, we'll use this ID as the target of our CLI command - in this case, "Modify Node Add Class" (see CLI Commands):
/mnac 131959292244, SpecialClickableTreeClass
This can be done for any node (whether it's visualized by the client or not) and any DOM definition (depending upon the CLI command). This is a quick and easy method of examining and modifying individual object instances without resorting to creating complex HSL scripts.
Example 3: IDs in HSL
Most commonly, IDs in HSL are acquired through the casting of NodeRefs to the ID type. Without the hard reference to the node to which it refers, these IDs may be passed between the server and the client (or between areas) without risk of the reference being broken.
A simple example of this behavior is the submission of data from one server instance to another, where a node that is loaded in one area (such as a player) is not loaded in the other. Communicating data about an alien node like a player instance would be communicated like this:
method notifyAreaInstanceOfPlayerPresence(areaID as ID, areaInstanceID as ID, playerID as ID) call area areaID instance areaInstanceID $PLAYER_TRACKING_SYSTEM.receiveNotificationOfPlayerPresence(playerID) . method receiveNotificationOfPlayerPresence(playerID as ID) MsgArea("debug", "Notification of " + playerID + " received! Player seen in area " + SYSTEM.REMOTE.FROMAREA + ":" + SYSTEM.REMOTE.FROMINSTANCE )) .
Other HSL tools
external function CreateID() as ID
The external function CreateID() exists to offer more fine-grain control over the allocation of an area instance's IDs. An ID returned by this external function is guaranteed not to conflict with any other instance and may be used as a global unique reference for data stored outside of the database by a game system.
public function RequestIDFromIDSpace( IDSpaceName as String, callbackScript as ScriptRef, callbackFunction as String, args as LookupList indexed by String of String ) as ID
In addition to the CreateID() external function (which allocates 'real' IDs from the global id pool), there exists a set of tools (IDSpaceUtils) to reserve 'non-real' IDs from arbitrary pools. These IDs have no relation to those assigned by the ID server from the global pool, but they have the benefit that they do not consume global IDs, they begin from a more intuitive count of 1, and they are entirely under HSL control. These tools may be used internally by game systems to keep track of data without consuming global IDs.
requestID as ID // ID spaces for oracles are the same name as their classes requestID = IDSpaceUtils:RequestIDFromIDSpace( "AbilitySpecOracle", SYSTEM.EXEC.THISSCRIPT, "onNewIDReturned", args )
remote function onNewIDReturned( rmc as class RemoteCallIn ) requestID as ID = rmc.args["requestID"] IDSpaceName as String = rmc.args["IDSpaceName"] newID as ID = rmc.args["newID"] // Do whatever your system needs to do with the new ID, note the requestID is passed back // so you can match it up with the requestID that was supplied when you made the request. .
Using SQLPlus to change the NEXT_ID Line
The following documentation is intended for customers running their own servers.
In the database, there is a schema called FS_ID which has a table ID_LINE. The ID_LINE table has the all important NEXT_ID, BATCH_SIZE and MINIMUM_PERCENT values for each ID line. When spinning up a new production world, it should be configured (in master control) to either utilize a shared id line for all worlds in that universe which you initialize to a value no less than the lower bounds ID for your production universes or each individual production world's id line is initialized to the minimum lower bound.
Assuming that your ID schema is still called FS_ID (and it would be unless you've made changes to make it not be), this SQL line will set the next ID space to 13835058055282163712, which is half-way through the ID space allocated to customers. (this is the magic number 2^63 + 2^62 ).
From the configured value for HEBatchIDLine (where ours is HE_DEV, and I don't know what yours is), replace that in the 'XXXXX' below.
SELECT * FROM FS_ID.ID_LINE WHERE LINE_NAME='XXXXX';
To Update only that line to initiate allocation from the new point onward, replacing XXXXX with your value of he_dev, and magic_number with the magic number above (which partitions the space in half):
UPDATE FS_ID.ID_LINE SET NEXT_ID = magic_number WHERE LINE_NAME = 'XXXXXX';
Please note, that the SQL editor you use will determine how you make the full contents of the columns display correctly. If you are using SQL*Plus, do this:
column next_id format 999999999999999999999999 select * from fs_id.id_line where line_name = 'XXXXX';
... confirm it shows 922337...etc... in NEXT_ID
update fs_id.id_line set next_id = magic_number where line_name = 'XXXXXX'; select * from fs_id.id_line where line_name = 'XXXXX';
... confirm it shows magic_number, if so:
... else cancel the transaction: