ID

From HEWIKI
Revision as of 19:26, 10 July 2012 by HE-CHRISTOPHER (Talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

An 'ID' is an unsigned 64 bit integer used to identify GOM and DOM data in the database both locally and on the server. Each GOM node or DOM definition is linked to a unique 'ID' and will never be re-used (even if it is deleted).

Contents

Overview

Figure 1. ID Pathway Diagram

'ID's are used by the engine to uniquely identify persisted and non-persisted instances as well as to identify class and field definitions within the DOM. These 'ID's are managed by an ID Server, which is responsible for distributing blocks of IDs to Area Server instances so that they may facilitate edit requests for the creation and manipulation of object instances. Each object instance corresponds to one and only one 'ID': a contract which the ID Server enforces by maintaining sole responsibility for distributing new blocks of IDs to area servers. IDs may also be acquired and assigned via HSL script, either explicitly through the use of HSL ID Spaces (which come from separate pools of IDs) or implicitly through the creation of persisted or non-persisted objects. IDs are also used heavily in the CLI to effect changes to DOM definitions. Finally, since the space from which IDs are assigned has a size of approximately 2^64, the risk of a world exhausting all possible IDs is very small.

What problem(s) does this solve?

What problem(s) does this not solve?

Concepts

The relationship between IDs, areas and the ID server is illustrated in Figure 1. The ID Server - being the sole producer of IDs - is responsible for fulfilling requests by area servers who are in need of IDs. It does this by reserving ranges of IDs from the database and distributing these IDs to area servers in need. The area server - once supplied - uses the provided IDs to facilitate creation of persistent and non-persistent nodes as well as DOM definitions.

Distribution

Each world has an ID Server which is responsible both for managing the set of IDs currently in use in the world as well as reserving and distributing new IDs to Area Servers as they make requests for them. Upon receiving such a request, the ID server will reserve a block of IDs of some fixed (configurable) size and deliver it to the requesting area server. These IDs will be pulled from the ID line associated with the current world and their values will increment starting from the value of the end of the previous block and spanning the range indicated by the currently configured block size. This process is illustrated in Figure 2:

Figure 2. ID Acquisition
IDs may be requested by the area server in one of two situations:
  • The area server is just spinning up and requires an initial set of IDs with which to begin creating nodes (persistent or otherwise)
  • The area server has already allocated some percentage (configurable) of its available IDs and is at risk of exhausting its current supply

In either case, the area server will fire off a request to the ID Server for a new block of IDs. Upon fulfillment of these requests, the area server will have access to an additional block of IDs with which to assign nodes.

Note: It is entirely possible for an Area Server to exhaust all of its IDs before the ID Server can deliver (asynchronously) a new batch. In a normally functioning world, this is an extremely unlikely event, but an out-of-control script or incredibly dense area import can cause issues. Solutions to such an incident, should it occur, include performing smaller asynchronous operations in-script and increasing the block size for allocated IDs.

Allocation

When an area server spins up and receives its block of IDs, it is then capable of handling requests from the editor (or script) to create new nodes and DOM definitions. These entities are allocated IDs incrementally starting with the least-most available in the area's list of assigned IDs. As entities are created and IDs are consumed, the list of available IDs will shrink until - at some point - a new batch is requested from the ID Server. This then refreshes the local store of IDs and the process may continue.

An area that shuts down prior to using all of its assigned IDs will drop all of its remaining IDs and they will be lost. There is no aggregation and recycling of unused IDs, as this would be extremely inefficient to track and manage. For large multi-world universes where collisions become more likely, other considerations need to be taken into account (see Partitioning below).

Partitioning

In multi-world setups (e.g. development world, QA world, multiple production worlds, etc), collisions of IDs must be strictly avoided. If changes are made in one world that begin consuming IDs that were previously assigned to the production world, the ID spaces begin to overlap and world data becomes corrupted. To avoid this, we can partition the ID space such that no reasonably-sized world will consume more IDs than are contained within its designated partition. This ensures that any push of data from one world to another does not create an irrecoverable corruption by overwriting or otherwise impinging upon the ID space of the destination world.

Partitioning need take only the following into account:

This is illustrated in Figure 3:

Figure 3. ID Space Diagram


In order to guarantee no collisions between ID spaces, we need to make the dev ID space (or - alternatively - the ID space of any world from which you might someday wish to initiate a push of data) large enough to contain any possible 'reasonably sized world'.

Determining the total ID space requirements for a development world is left as an exercise for the customer and involves the following criteria:


Again, and this bears repeating:

It is critical that production servers not allocate IDs in a range overlapping the development server(s) if the ID Line is not shared. Failure to address this will result in an inability to publish development content to the production worlds.


The following table describes the manner in which the ID Space is partitioned by the HeroEngine team.


LOW HIGH OWNER PURPOSE
0x0000_0000_0000_0000 0x0000_0000_7FFF_FFFF
2147483647
HeroEngine Legacy Definitions, Areas
0x0000_0000_8000_0000
2147483648
0x0000_0000_FFFF_FFFF
4294967295
HeroEngine Client-side temporary IDs
0x0000_0001_0000_0000
4294967296
0x0FFF_FFFF_FFFF_FFFF
1152921504606846975
HeroEngine Legacy Definitions, Areas
0x1000_0000_0000_0000
1152921504606846976
0x3FFF_FFFF_FFFF_FFFF
4611686018427387903
HeroEngine HeroEngine Base Clean Packages
0x4000_0000_0000_0000
4611686018427387904
0x5FFF_FFFF_FFFF_FFFF
6917529027641081855
HeroEngine HeroEngine Clean
0x6000_0000_0000_0000
6917529027641081856
0x7FFF_FFFF_FFFF_FFFF
9223372036854775807
HeroEngine Template Worlds
0x8000_0000_0000_0000
9223372036854775808
0xFFFF_FFFF_FFFF_FFFF
18446744073709551615
HeroEngine HeroEngine Full Customers
0x8000_0000_0000_0000
9223372036854775808
0x8DFF_FFFF_FFFF_FFFF
10232178353385766911
HeroCloud HeroCloud Development Worlds
0x8E00_0000_0000_0000
10232178353385766912
0x8FFF_FFFF_FFFF_FFFF
10376293541461622783
HeroCloud HeroCloud Staging Worlds
0x9000_0000_0000_0000
10376293541461622784
0xFFFF_FFFF_FFFF_FFFF
18446744073709551615
HeroCloud HeroCloud Production Worlds

Configuration

The following quantities may be configured for IDs:

Usage

IDs make creating, accessing and modifying data very straightforward. They can be used explicitly to reference a node or definition via HSL or CLI commands, and they are also embedded into visualized nodes in HeroBlade.

To illustrate the major methods of using IDs, let's go through a few simple examples:

Example 1: IDs in HeroBlade

Figure 4. HeroBlade IDs

Let's say we have an area that we've been building for quite some time now. It contains dozens of asset instances, a couple of NPCs and various regions, triggers and pathfinding nodes. Now - selecting any of these assets (let's just say a Tree) and bringing up the Properties panel will reveal a field named GUID. This is the unique ID assigned to the selected instance and may be used as a reference for use in HSL scripts or CLI commands. Whenever a command is issued with one of the built-in transformation widgets or an asset instance is added via the library, this ID is used internally to act as a reference for the node in question. Each node visualized in the HeroBlade client is associated with one of these Global Unique IDs.

In this case, our tree instance has an ID of 131959292244.

Example 2: IDs in the CLI

Now, let's take our tree that we selected in the previous example. Let's say we want to modify its properties from the CLI (perhaps we want this to be a tree that has additional functionality built into it that we can't visually inspect or edit with HeroBlade). In order to do so, we first need to know the ID of the instance; this is something we discovered in the first example.

Now, we'll use this ID as the target of our CLI command - in this case, "Modify Node Add Class" (see CLI Commands):
/mnac 131959292244, SpecialClickableTreeClass

This can be done for any node (whether it's visualized by the client or not) and any DOM definition (depending upon the CLI command). This is a quick and easy method of examining and modifying individual object instances without resorting to creating complex HSL scripts.

Example 3: IDs in HSL

Most commonly, IDs in HSL are acquired through the casting of NodeRefs to the ID type. Without the hard reference to the node to which it refers, these IDs may be passed between the server and the client (or between areas) without risk of the reference being broken.

A simple example of this behavior is the submission of data from one server instance to another, where a node that is loaded in one area (such as a player) is not loaded in the other. Communicating data about an alien node like a player instance would be communicated like this:

method notifyAreaInstanceOfPlayerPresence(areaID as ID, areaInstanceID as ID, playerID as ID)
  call area areaID instance areaInstanceID $PLAYER_TRACKING_SYSTEM.receiveNotificationOfPlayerPresence(playerID)
.
 
method receiveNotificationOfPlayerPresence(playerID as ID)
  MsgArea("debug", "Notification of " + playerID + " received! Player seen in area " + SYSTEM.REMOTE.FROMAREA + ":" + SYSTEM.REMOTE.FROMINSTANCE ))
.

Other HSL tools

external function CreateID() as ID

The external function CreateID() exists to offer more fine-grain control over the allocation of an area instance's IDs. An ID returned by this external function is guaranteed not to conflict with any other instance and may be used as a global unique reference for data stored outside of the database by a game system.

public function RequestIDFromIDSpace( IDSpaceName as String, callbackScript as ScriptRef, callbackFunction as String, args as LookupList indexed by String of String ) as ID

In addition to the CreateID() external function (which allocates 'real' IDs from the global id pool), there exists a set of tools (IDSpaceUtils) to reserve 'non-real' IDs from arbitrary pools. These IDs have no relation to those assigned by the ID server from the global pool, but they have the benefit that they do not consume global IDs, they begin from a more intuitive count of 1, and they are entirely under HSL control. These tools may be used internally by game systems to keep track of data without consuming global IDs.

requestID as ID
// ID spaces for oracles are the same name as their classes
requestID = IDSpaceUtils:RequestIDFromIDSpace( "AbilitySpecOracle", SYSTEM.EXEC.THISSCRIPT, "onNewIDReturned", args )

remote function onNewIDReturned( rmc as class RemoteCallIn )
  requestID as ID = rmc.args["requestID"]
  IDSpaceName as String = rmc.args["IDSpaceName"]
  newID as ID = rmc.args["newID"]
 
  // Do whatever your system needs to do with the new ID, note the requestID is passed back
  //   so you can match it up with the requestID that was supplied when you made the request.
.

See also

Reference

Using SQLPlus to change the NEXT_ID Line

The following documentation is intended for customers running their own servers.

In the database, there is a schema called FS_ID which has a table ID_LINE. The ID_LINE table has the all important NEXT_ID, BATCH_SIZE and MINIMUM_PERCENT values for each ID line. When spinning up a new production world, it should be configured (in master control) to either utilize a shared id line for all worlds in that universe which you initialize to a value no less than the lower bounds ID for your production universes or each individual production world's id line is initialized to the minimum lower bound.


Assuming that your ID schema is still called FS_ID (and it would be unless you've made changes to make it not be), this SQL line will set the next ID space to 13835058055282163712, which is half-way through the ID space allocated to customers. (this is the magic number 2^63 + 2^62 ).

From the configured value for HEBatchIDLine (where ours is HE_DEV, and I don't know what yours is), replace that in the 'XXXXX' below.

 SELECT * FROM FS_ID.ID_LINE WHERE LINE_NAME='XXXXX';

To Update only that line to initiate allocation from the new point onward, replacing XXXXX with your value of he_dev, and magic_number with the magic number above (which partitions the space in half):

 UPDATE FS_ID.ID_LINE
   SET NEXT_ID = magic_number
   WHERE LINE_NAME = 'XXXXXX';

Please note, that the SQL editor you use will determine how you make the full contents of the columns display correctly. If you are using SQL*Plus, do this:

 column next_id format 999999999999999999999999
 select * from fs_id.id_line where line_name = 'XXXXX';

... confirm it shows 922337...etc... in NEXT_ID

 update fs_id.id_line set next_id = magic_number where line_name = 'XXXXXX';
 select * from fs_id.id_line where line_name = 'XXXXX';

... confirm it shows magic_number, if so:

 commit;

... else cancel the transaction:

 rollback;

Personal tools
Namespaces
Variants
Actions
Navigation
Toolbox