Data storage options

From HEWIKI
Jump to: navigation, search

Contents

Design Considerations for Persisted Node Data Storage

Overview

This page details the many ways a scripter can persist (save) information on the server for the implementation of game mechanics and systems. There are many different ways to do this, as each has their pro and con tradeoffs. This page will guide you through the various options and explain when each is appropriate to use.

In general, every persistent node in the GOM is saved in the database. This represents the most fundemental way in which data is saved. However, one must consider how to get access to that node later to retrieve the data. Further, there are other ways to save data that might be more appropriate than this. This page will walk you through all of the HeroEngine features at your disposal for persisting data.

Granularity

Several of the data storage options detailed on this page require the creation of nodes to hold the data you want to persist. When a persistent node's fields are changed, this will ultimately cause a write to the database at some point. It is critically important to carefully control the amount of database "hits" that go on; too many and the server processes will begin to slow and eventually backup. Because of this some things must be carefully considered when using persistent nodes:

If you have, for example, a top level field field myBigComplexField on a persisted node that is a list or lookuplist of class y, which in turn is composed of a lookuplist of class z and some other fields... you need to be aware that ANY changes to a cell to a field in class z causes the entire field myBigComplexField to be serialized and saved to the database. Depending on the number of elements and frequency of write accesses, this can have extremely bad performance implications.



For applications that may require extremely frequent write access to a persisted node, it is best to make the write accesses as granular as possible.

Granularity 
(attrib: The Free On-line Dictionary of Computing, 1993-2004 Denis Howe) The size of the units of code under consideration in some context. The term generally refers to the level of detail at which code is considered, e.g. "You can specify the granularity for this profiling tool". The most common computing use is in parallelism where "fine grain parallelism" means individual tasks are relatively small in terms of code size and execution time, "coarse grain" is the opposite. You talk about the "granularity" of the parallelism. The smaller the granularity, the greater the potential for parallelism and hence speed-up but the greater the overheads of synchronization and communication.


So taking our original field mybigComplexField, how do we fix it to be more granular?

Well, the first thing we should do is make the lowest level of related data (class z in this case) into its own persisted node. The field that previously was a lookuplist of class z is changed to be either a lookuplist of noderefs, or you can simply do away with it entirely and use the association engine.

Whether you use the association engine as the "list of class z's" or not, you must make a base hard association to the node of class z to make it load with the character or area. You could make additional soft associations to qualify the relationship more accurately. Depending on the number of persisted class z nodes you may find that looping through the list of associations to be less than efficient, in this case a lookuplist map is probably ideal.

This is great, but now what about class y? Well we can do the same thing to the lookuplist of class y, make persisted nodes out of the class y list objects and make associations or a lookuplist map of noderefs.

The end result of this is that changing a field on one of the class z nodes ONLY forces the serialization and writing of THAT field on THAT specific class z node.

Scope

When referring to the data's scope, it is a reference to where the data "exists" and what has access to read/write to the data. There are several possibilities for the scope: Local GOM, Global (all server GOMs), Local Client GOM, Client and Server. When talking about the scope of data, we do not include Inter-Process Communication because it blurs the line into everything being global.

Local GOM

Each instance of an area has what is called a Local GOM, it is here that all nodes in the area are loaded into memory. Persisted nodes may be loaded in only one GOM at any given time, it is that GOM that is permitted to make persisted changes. Non-persisted nodes may be altered in a local GOM, but those changes are neither reflected nor persisted in any other local GOM.

Play instances have no persisted nodes in them other than the account (and by extension the character and any nodes associated to it) which is what is called a "root" node. The area's root node is a non-persisted node in play instances.

Global

Data that is global in scope is available to all Local Server GOMs. However, several of the methods that provide global access to data have restrictions on write operations.

Local Client GOM

The client GOM starts with only the most basic information populated by HeroEngine to establish the minimum level of functionality. In practice, that means that the client knows the assets in an area and their properties (not to be confused with fields you may have added to an asset on the server via GLOMming) and any nodes scripts may have created.

Changes to nodes in the client GOM are not reflected to the server. Game systems often cache data in the client GOM to avoid round-trip communications with the server and expose interfaces for working with that data to communicate changes to the server if needed.

Client and Server

Several of the more specialized storage techniques have a scope of Client and Server, which means that both client and server have the opportunity to access the data. These methods all have restrictions on write access.

Technical Difficulty

The technical difficulty of a storage technique is meant as a gauge of the general level of scripting proficiency required to implement a system using that technique.

Read Access

Speed

The speed at which data may be read varies for the differing storage techniques.

Type

Access to the data may be Asynchronous, Synchronous, or a mixture where the first access is asynchronous and the data is cached to make subsequent access synchronous.

Write Access

Speed

The speed at which data may be written varies for the different storage techniques.

Type

Access to to the data for the purpose of writing may be Asynchronous, Synchronous or Disallowed.

Crash Tolerance

Crash tolerance refers to how well the data is capable of surviving a crash. For most methods of data storage this is mainly dependant upon the database and thus not an issue scripters need concern themselves overly.


Memory Footprint

Sometimes the choice of storage techniques is limited by the memory footprint of the data to be stored. Not all storage techniques are equally valid for all ranges of memory footprints. This is meant to be a general gauge as to the appropriate size of a data set, there is no hard and fast rule that must be followed.

However, even though there is no hard rule about what footprint is appropriate for a particular method of storage there are absolute limitations imposed by your hardware. For example, you simply can not load more objects into memory than you have available RAM.

Queryability

Queryability refers to how easy it is for other systems to access the data. Some storage techniques do not or can not expose the data except by explicitly adding code to do so.

Scaleability

Scaleability refers to how well a particular storage technique is suited to handling extremely large numbers of customers. This is often achieved by accepting compromises in other areas such as the read/write access, scope, memory footprint and so on.

See also: Node Persistence


Global Persisted Data Storage Options

Overview

A HeroEngine game is composed of many Area instances which are all individual processes. There needs to be a way for all of these area instances to store and retrieve data.

HeroEngine provides several potential repositories for data: Scripts, Prototypes, Node Delivery Service, System Areas, the World Server and the Repository. Of these, only System Areas (and by extension the World Server) are suitable for data that must have read/write access in all instances in a production (live) world. For data that only needs to be read, there are many more options for data storage.

Data Storage in Scripts

Summary

Storing data in script makes sense for a system when:


Scripts are particularly suited for storing what are in effect constants. Whether that is a simple boolean to a complex class variable. Data stored in scripts is not modifiable from HSL and can not be modified dynamically. Consequently, scripts are unsuited for storing data that needs to be modified via GM tools.


Advantages Disadvantages
Technical Difficulty low Ability to Handle Large Data Sets low
Data Read Access extremely fast
Synchronous		 Data Write Access none
Data is static
Data can not be modified via HSL
Supported Frequency of Data Access high Queryability low
Support must be coded for each query
Crash Tolerance extremely high    
Scope global or local client GOM
Server scripts have a global scope
Client scripts have a scope of local client GOM		    


Design

Storing data in a script has a low design requirement, you must be able to call a function in the script and have it return the data in a format your script is able to interpret. If you intend other scripts to be able to access the data stored in your script, the access point (function) must be declared as public.

Storing data in a script does require that the data be copied into a variable (scalar or class variable), as with any computer language copying data is relatively slow operation. Consequently, scripts are poorly suited for large datasets. 

public function myData() as lookuplist indexed by string of string
  myList as lookuplist indexed by string of string
  myList["Blue"] = "Blue is not the color of my house."
  myList["Red"]  = "Red is a color found in our flower bed."
  myList["Green"] = "Green is sometimes the color of our grass."
  return myList
.

Of course it is also possible to create an extremely complex class, populate it with all of the data it needs in your script and return it. 

Class:  MyReallyComplexClass
  Field:  AreaInstance as class InstanceKey
                      .AreaID
                      .AreaInstanceNumber
  Field:  ComplexInfo as class InfoTable
                     .InfoLookupList
                     .InfoList
                     .InfoRequesterAccountID
                     .InfoTable
  Field:  AccountID as ID

public function myComplexData() as list of class MyReallyComplexClass
  myReallyComplexClassList as list of Class MyReallyComplexClass
  // Fill out the data for 0...n MyReallyComplexClasses and add them to a list
  return myReallyComplexClassList
.


Example

Scripts provide a good place to store constants that do not need to change dynamically. During development of Hero's Journey we needed to test different types of creature navigation in and out of combat, such as snapping rotation vs animating the turn. Rarely changing configuration values like this could be stored in a prototype, but doing so would really be overkill when the desired behavior is the ability to swap between methods during development. 

#define snaprotation
 
public function SnapRotation() as boolean
  #if snaprotation
     return true
  #else
     return false
  #endif
.
 
public function DoTurn()
  if SnapRotation()
    // Just snap the character
  else
    // Send behavior to client to turn the character
  .
.

While this represents a simplistic example, it would be equally valid to return a class variable rather than a boolean.

Data Storage In Prototypes

Summary

Storing data in prototype makes sense for a system when:


The DOM allows us to construct classes and their fields, but it does not permit the setting of a default value. Prototypes provide the mechanic by which we can set up objects with default values. Prototypes are loaded into the GOM of an AreaInstance during its spinup, meaning they are in memory for the AreaInstance to read.


Advantages Disadvantages
Technical Difficulty medium Data Write Access None/Asynchronous
No write access permitted in production(live) worlds
Slow write access in the development environment
Writing to a prototype is a coordinated DOM change
Data Read Access extremely fast
Synchronous		 Memory Footprint varies
Queryability high
Data is exposed to all systems equally		    
Crash Tolerance extremely high    
Scope global or local client GOM
Server prototypes have a global scope
Client prototypes have a scope of local client GOM		    


Systems that Use Prototypes for Data Storage


Design

As far as HeroScript is concerned, prototypes are handled very similarly to how you handle a node.

While prototypes are loaded on demand by HeroEngine, prototypes in constant use have a memory footprint equal to their footprint times the number of servers on a physical machine. Consequently, consideration must be given as to the memory footprint a prototype will occupy.


Gotcha's

Modifying a prototype triggers a coordinated change in the DOM, for this reason writing to prototypes is NOT permitted in code that will run in production(live). Consequently, prototypes are best for Global Data Storage that GameMasters will need to modify easily (via some GM verb) but ONLY in DEV.

Coordinated Change 
Coordinated DOM changes freeze processing in ALL AreaInstances of a World until the update is complete.

Note: The coordinated change happens at the end of script execution for any fields marked "dirty". This means Sorting a list on a prototype does not cause any more coordinated events than is absolutely necessary.


Reference a Prototype

Getting a reference to a prototype is achieved through the use of the external function getPrototype(name as string) as noderef. 

myPrototype as noderef = getPrototype( "myPrototypename" )


Reading a Prototype

Reading a prototype to which you already have a reference is exactly like accessing the value of a field on a node. 

myString as string = myPrototype.Name


Modify a Prototype

CautionSign.gif It is NEVER permissible to modify a prototype in the production(live) worlds Contact your technical support list if you think you need to do so and we will come up with an alternative.


Modifying a prototype to which you already have a reference is just like changing a field on a noderef. 

myPrototype.myField = 100000


Example

Games commonly have what are referred to as "slash commands" or "verbs"; when sent via chat to the server such commands execute some sort of logic. The HeroEngine CommandHQ handles registering commands with the code that is invoked when the command is run.

For ease of use, it is convenient if adding a new command does not require changes to the CommandHQ code. Add/Remove operations only occur during development so the ability to modify the data in a production world was not needed. Commands are typically contextual in nature making them sensitive to which instance the code is running, so the map of command to code needed to be available in all instances.

Consequently, a prototype "ProtoCmdMaster" was choosen as the repository for the data for the command system.

Data Storage In Specs (MMO Foundation Framework)

Summary

Storing data in a spec makes sense for a system when:


Specs are the marriage of several of the storage techniques detailed on this page. Immutable (unchanging) data is stored in prototypes on the server. The prototypes know how to write themselves out to repository files providing for an efficient transmission mechanism to the client that occurs only when a file changes. Objects instantiated from a spec are then stored as nodes dangling off of a root node such as the character or area root, it is these nodes that store mutable (changing) data that is transmitted to the client via Inter-Process Communication as needed.


Advantages Disadvantages
Technical Difficulty low Data Write Access None/Asynchronous
No write access permitted in production(live) worlds
Slow write access in the development environment
Writing to a prototype is a coordinated DOM change
Data Read Access (server) extremely fast
Synchronous		 Memory Footprint varies
Specs are loaded on demand.
Queryability high
Data is exposed equally to all systems		 Data Read Access (client) varies
Asynchronous
Slow I/O limited access for first read, subsequent requests are fast as the spec is cached in memory.
Crash Tolerance extremely high    
Scaleability extremely high
Reducing memory usage for shared data by referencing a template is critical to systems that have lots of objects with some amount of immutable data.		    
Scope client and server    


Systems that Use Specs for Data Storage


Design

The spec system is a Required System included in HeroEngine, consequently there is a very low design requirement as most of the functionality is already in place. Detailing the specifics of using the spec system is beyond the scope of this page.


See also: Spec Oracles



Example

There were originally two basic strategies for the implementation of items in an MMOs:

There are performance issues with every item being a unique copy of its data, and there are design issues with all items being references to static templates. The limitations of the two strategies as absolutes lead to the more common implementation in todays MMOs of a blend between the two. As much immutable data as is possible is stored in a template and mutable data is stored as a unique instance of the data specific to a particular item.

Items in Hero's Journey are implemented as SpecDerivedObjects, which have a reference to a spec that serves as the repository for their templated data and mutable data stored as a part of the item itself.

Local Server Persisted Data Storage

Overview

Data storage for a local server is much easier than global data storage. You can store the data directly on the area root node or in nodes associated via hard associations to the root.

The biggest disadvantages are the limited availability of the information and potential for the area in which the data resides being down.

Data Storage with the Area Root Node

Summary

Storing data with the area root makes sense when:


Advantages Disadvantages
Technical Difficulty low Memory Footprint varies
Some consideration should be given to the memory footprint of the data.
Data Read Access extremely fast
Synchronous 		Ability to Handle Large Data Sets low
Data exists for each instance of the area, this becomes a potential scaleability issue.
Supported Frequency of Data Access high Scope local
global Via Inter-Process Communication
Possibile the area is not spun up
Crash Tolerance high Data Write Access varies
Writing to a local node does not update all areainstances, nor does the data persist if the instance is not the edit instance.


Design

There are several potential methods of storing the data with the area: creation of a node associated to the area root or a class that is GLOMmed onto the area root node.

Storing the data in a node requires that you; Instantiate a persisted node from a class containing the field(s) you need to store your data and associating that node to the area root node in the EDIT instance of your area. The node(s) must be associated to the area root with at least a base_hard_association and may optionally have additional soft associations to facilitate querying for the node(s). Using nodes to store the data has an advantage over storing the information in a class GLOMmed on, in that you may have multiple nodes to store data should your system require them.

Storing the data in a data class GLOMmed onto the area node is trivial to implement but has a disadvantage in that the root node becomes a harder to work with as it has more classes added to it increasing the odds of method conflicts and conflicts with how a particular field is used. Please note, when GLOMming it is important to make sure you are not reusing fields because we do not have multiple inheritance. (Or rather we do sort of, in that it is fine to inherit the same field from multiple classes, but you end up with just ONE of those fields leading to the possibility of multiple systems using the same field differently...very bad).


Example

Monster generators (aka Mongens) contain data that is specific to a particular area. Mongens are set up in the edit instance of the area as persisted nodes that are associated to the area root node with a base_hard_association. Consequently, those mongens exists in all play instances of a particular area, any changes in one play instance are local and are not persisted so that timers can fire, counters can increment all without changing the persisted version in the edit instance. In this way, we are guaranteed that each new instance of the area is set up properly in its default state.

Data Storage in System Areas

Summary

Storing data in a System Area makes sense for a system when:

A System Area is essentially an Area server that is dedicated to some sort of processing rather than managing players and geography. An Area can be designated to handle some particular game mechanic, system or data store. Additionally, multiple instances of a System Area can be spun up to dynamically distribute load.

System Areas are a clever way of utilizing the instanced nature of HeroEngine's Area servers to implement extremely scaleable systems and offload large data sets or computationally intense systems from other servers. The World Area instance is a special type of System Area, but there can only be one World Area server. Custom Area servers can be utilized to implement any sort of parallel processing that is required.


Advantages Disadvantages
Scaleability Very High to Extremely High Technical Difficulty High to Very High
All of the common problems associated with database programming, such as collisions.
Asynchronous programming
Data Read/Write Access very fast(r) fast(w)
Asynchronous
Access speed is dependent upon Inter-Process Communication speed (x2) plus the time required to query the data in the System Area via script.		 Crash Tolerance varies
Low to very high dependant on the skill of the system's coder.
Ideally, system areas must be able to seamlessly recover from crashes
Supported Frequency of Data Access high to very high    
Queryability high
By the very nature of system areas, built-in support for querying data in the system area is almost a requirement.		    
Ability to Handle Large Data Sets very high    
Scope local GOM
Systems created using a system area by their nature require they expose easy to use interfaces. Consequently, the local scope is not a disadvantage from the perspective of a user of the system.		    


Systems that Use System Areas for Data Storage


Design

System Areas communicate with normal areas through Inter-Process Communication. They do not handle users directly as do normal Areas.

The creation of a properly designed System Area is probably one of the most challenging scripting tasks possible in the HeroEngine environment. System areas mix the complexities of asynchronous environments with those of database design. Plus, often there is a need to implement a Client GUI allowing GameMasters to easily manipulate System Areas (or the data they manage).

Although complex, System Areas allow for you to design arbitrary units of computation that can scale well.

See also: Asynchronous Considerations



Example

Hero's Journey has an extremely complex quest system that assembles virtually unique quests for characters based on a large number of factors. This requires a significant amount of processing time as well as a very large data set from which a fully fledged quest is assembled. Those factors required the implementation of the first and most complex system area.

The quest system area acts as a repository for the quest part specifications and the processor in charge of assembling one or more potential quests for a character. System scaleability is achieved by the implementation of the edit instance as a router/administrator for 1...n play instances of itself. As the system load changes, the edit instance is able to spin up/down play instances and adjust its routing of requests to balance the load.

The play instance, each its own thread, handle the processing for querying and assembling quests for a particular character.

Data Storage at the World

Summary

Storing data in the World (area 0, instance 0) makes sense for a system when:


Typically the only data stored at the world level is data that is required for the world to handle the very limited functionality implemented there. When implementing a system that is going to store data in the world area instance, an evaluation should be performed to determine whether or not that data really belongs there rather than some other data storage option.


Advantages Disadvantages
Technical Difficulty Medium Memory Footprint restricted to low by policy
Data Read/Write Access very fast(r) fast(w)
Asynchronous
Access speed is dependent upon Inter-Process Communication speed (x2) plus the time required to query the data in the world via script.		 Ability to Handle Large Data Sets low
Data occupies memory in the world server and uses processing time to query the data.
Supported Frequency of Data Access medium Queryability varies
The data is available on the world server to any process via Inter-Process Communication, but does not have built-in support unless a coder added it.
     Processing Restrictions Processing required to work with your data on the world must be minimal in nature to ensure the scaleability of the world server.
     Scope local GOM
global via Inter-Process Communication


Design


Assuming you understand asynchronous programming and Inter-Process Communication, storing data at the world level is a fairly simple procedure. You must create a persisted node from some class that has the field(s) you need for data storage. The node must be associated with the worldroot via at least a base_hard_association.

To access/write information you will perform Inter-Process Communication to the world to your script and perform whatever updates you wish to persist or retrieve whatever information you need.

Optionally, you could GLOM your dataclass onto the world root node directly.


See also: Asynchronous Considerations



Example

The travel system handles moving characters from one instance to the next, on occasion the destination instance is not currently running. To handle the case where an instance is not yet ready for a character, the travel system stores a queue on the world of pending travel requests. When an instance spins up, it checks the queue to see if there are any pending requests and processes them.

The world area instance is ideally suited to this particular data storage because the callbacks from the C++ Engine happen in the world area instance.

Data Persisted for a User

Overview

Data storage with the Account Node

Summary

Storing data with the account makes sense when:


Data stored on or associated to the account node should be data that is shared between all characters of the account. Any character specific information should be stored with the character node instead of the account node.


Advantages Disadvantages
Technical Difficulty low Memory Footprint varies
Some consideration should be given to the memory footprint of the data.
Data Read/Write Access extremely fast(r) fast(w)
Synchronous 		Ability to Handle Large Data Sets low
Data probably exists for each account, this becomes a potential scaleability issue.
Crash Tolerance high Scope local GOM
Only accessible when the account is connected.
global Via Inter-Process Communication


Design


There are several potential methods of storing the data with the account: creation of a node associated to the account node or a class that is GLOMmed onto the account node.

Instantiate a persisted node from a class containing the field(s) you need to store your data and associate that node to the _playerAccount node. If necessary, you may wish to consider creating your own association definition.

GLOM your data class onto the character. Please note, when GLOMming it is important to make sure you are not reusing fields because we do not have multiple inheritance. In other words, if your class has the account_name field and you glom that onto the _playerAccount node that already has that field you can mess up other systems if you change the field's value to something other than what the system requires.

For many applications that are specific/atomic in nature, it is safer to create a persisted node and create an association when working with the root nodes.

Data storage with the Character Node

Summary

Storing data with a character makes sense when:


Advantages Disadvantages
Technical Difficulty low Memory Footprint varies
Some consideration should be given to the memory footprint of the data.
Data Read/Write Access extremely fast(r) fast(w)
Synchronous 		Ability to Handle Large Data Sets low
Data probably exists for each account, this becomes a potential scaleability issue.
Crash Tolerance high Scope local GOM
Only accessible when the character is loaded.
global Via Inter-Process Communication


Design

There are several potential methods of storing the data with the character: creation of a node associated to the node or a class that is GLOMmed onto the node.

Instantiate a persisted node from a class containing the field(s) you need to store your data and associate that node to the _playerCharacter node. If necessary, you may wish to consider creating your own association definition.

GLOM your data class onto the character. Please note, when GLOMming it is important to make sure you are not reusing fields because we do not have multiple inheritance. (Or rather we do sort of, in that it is fine to inherit the same field from multiple classes, but you end up with just ONE of those fields leading to the possibility of multiple systems using the same field differently...very bad).


Example

Character abilities in Hero's Journey are at some level customizeable by the player. The basic implementation of immutable data is in the form of a Spec (check out storage in specs on this page) and the mutable data is stored in a node derived from the SpecDerivedObject class. Each character has an abilities node which is associated to the character node via a base_hard_association. Each ability is in turn associated to the character's abilities node.

This provides for character specific storage of mutable data for a particular ability, while maintaining the smallest possible memory footprint by storing the immutable data in a spec.

Data Storage in the Repository

Overview

HeroEngine's repository is a directory of files that exist on both server and client. If the client does not yet have a file, or the file is invalidated due to the existence of a update, the server automatically streams the file on demand to the client.

The implementation of the repository mechanics make it ideal for storing data is well suited for being cached locally to minimize bandwidth consuption.

Storing a Node in the Repository

Summary

Storing data in a node in the repository makes sense when:


Advantages Disadvantages
Technical Difficulty medium Data Read Access varies
Initial read is an I/O operation (asynchronous) and is slow, subsequent reads are extremely fast assuming you keep a reference to the node.
Scaleability extremely high
Efficient transmission of rarely changing data.		 Data Write Access slow
Writing to the repository file is probably only valid in the development instance as it requires all of the clients redownload the file the first time they attempt to access the data.
Supported Frequency of Data Access high     
Scope client and server     


Design


See also: Repository Node Serialization


Storing Arbitrary Data in the Repository

Summary

Storing arbitrary data in the repository makes sense when:


Using the repository to store arbitrary data, it is possible to store anything that can be serialized into a string format. Depending on the complexity of the data, the serialization step may be a significant hurdle for implementation.


Advantages Disadvantages
Technical Difficulty medium Data Read Access varies
Initial read is an I/O operation (asynchronous) and is slow, subsequent reads are extremely fast assuming you keep a reference to the node.
Scaleability extremely high
Efficient transmission of rarely changing data.		 Data Write Access slow
Writing to the repository file is probably only valid in the development instance as it requires all of the clients redownload the file the first time they attempt to access the data.
Supported Frequency of Data Access high     
Scope client and server      


Systems that use the repository to store aribrary data:


Design

Consideration must be given to the processing time required to deserialize the data when the repository file is read into memory. String manipulation is, as in any other language, relatively slow compared to other operations. An obvious optimization is caching the information once it is read into memory in a node, associated to a root node in some well known fashion for subsequent ease of reference.

The linked page provides details on the technical implementation of a system storing arbitrary data in the repository.

See also: Repository Data Storage



Example

Specs (see storage in specs on this page) store templated data that needs to be shared with the client. It would be inefficient to transmit that data each time a client logged into the game, so specs instead write themselves out to files in the repository in string form. On the client when a spec is requested, the file is read into memory and cached in a node.

Node Delivery Service

Storing Data using the Node Delivery Service

Summary

Storing data using the node delivery service makes sense when:


At the most basic level, the node delivery serivce provides the mechanics for "mailing" a node between characters or servers. This has obvious applications for an for game systems like an auction system and in-game mail system.

Clever use of the node delivery service can significantly reduce the memory footprint of a character in memory, making this a potentially important tool in efforts to ensure maximum scaleability of a system that requires a large number of data nodes that do not always need to be loaded with the character.


Advantages Disadvantages
Technical Difficulty medium Data Read Access varies
Scaleability extremely high Data Write Access slow
Asynchronous
Writing data requires you first load the existing data node from a mailbox or create a new one and send it to a mailbox.
    Supported Frequency of Data Access low
Retrieving nodes stored in mail requires they be loaded from the database, this is slower than a normal external function.
    Scope local
global via Inter-Process Communication


Design

See also: Node Delivery Service



Example

Hero's Journey in-game mail system uses the node delivery service allowing characters to "mail" nodes such as a sword to another character regardless of whether or not that character is ingame.

Local Server Non-Persisted Storage

Overview

Storing non-persisted data locally is essentially the same as persisted data, the only signficant difference is working with non-persisted nodes instead of persisted nodes. However, there is an additional option availible for the storage of non-persisted data and that is system nodes.

System Nodes

Summary


Storing data in a system node makes sense when:

CautionSign.gif The prototype that represents the system node, like all prototypes, may not to be modified in a production(live) world. It is only the non-persisted copies that may be modified.


System nodes are a specialized type of prototype that the HSL compiler knows how to locate using a well-known address. When a system node is first referenced in a local, a non-persisted copy of the system node's prototype is created and thereafter used for all subsequent references. Each area server has its own copy of the data in a different non-persisted node permitting the local server to make any changes or anchor any supplementary nodes to it as needed.

The system node will have default values that are identical to the system node prototype's values when first instantiated. System nodes also serve as an object upon which you may call any methods that might exist for a particular system.


Advantages Disadvantages
Technical Difficulty medium Data Write Access varies
non-persisted write access to the local copy of the prototype is permitted
No write access permitted in production(live) worlds to the system node prototype
Slow write access in the development environment
Writing to a prototype is a coordinated DOM change
Data Read Access extremely fast
Synchronous		 Memory Footprint varies
Supported Frequency of Data Access high    
Queryability high
Data is exposed equally to all systems		    
Crash Tolerance very high    
Scope global or local client GOM
Server system nodes are global in scope
Client system nodes are local client GOM in scope		    


Design

The design for HSL implementations for HeroEngine required that licensee's be able to override and extend behaviors. Using an object oriented approach suited the design requirement but requires that a node exist upon which method calls may be made and as a repository for any data required by the system.

System nodes are based on a prototype, when a system node is first referenced in script (e.g. SYSTEM.NODE.BASECLIENT or short hand $BASECLIENT) the engine creates a non-persisted copy of the prototype in the local GOM and maps the SYSTEM.NODE.PrototypeName system variable to the id non-persisted copy. The non-persisted copy may then be modified without affecting the prototype (which is not allowed in production worlds) or the copies of any other GOMs.


See also: System Nodes



Example

Development of a clean HeroEngine for distribution to Licensees required provision for a licensee to override certain Required System functionality to implement whatever game-specific behavior is desired. System nodes, like SYSTEM.NODE.BASECLIENT, implement unique methods for the callbacks from the C++ engine that then process any required code and make calls to game specific implementations for extension/overriding behaviors.

For Hero's Journey, the game specific implementation of BaseClient is found in the HJBaseClientClassMethods script. The prototype named BaseClient, located in the server's persisted client DOM (use the | in the CLI to access this), has the HJBaseClient class GLOMmed on.


Taking a look at the _BaseClientClassMethods script's implementation of _Area_Load: 

unique method _Area_Load(areaID as ID, instanceID as ID, AreaName as String)
// Called when Area_Loading is done.
// Note that AreaName is the area's name according to the Spec panel in HeroBlade, not
//   the name according to the Organizer panel.
// Take away the splash screen.
 
  me._SetAreaName( areaName ) // This always happens because some Required Systems
                              //   use the area name
 
  handled as Boolean
  if hasMethod( me, "HE_Area_Load" )  // Game-specific implementation
    handled = me.HE_Area_Load( areaName )
  .
 
  if not handled
    // default implementation(if any) if not 
    //   handled or the game specific implementation
    //   does not exist
  .
.


This example does not demonstrate a pattern likely to be duplicated in systems that are not part of the HeroEngine Required Systems, it does demonstrate how you can use system nodes as objects upon which methods may be called and non-persisted data may be anchored (via associations). The implementation of SYSTEM.NODE.Name and $Name enables a scripter to use the nodes without knowing or caring what the ID of the node might be in this particular local GOM.

Arbitrary Root Nodes

See also: Arbitrary Root Node
Retrieved from "http://hewiki.heroengine.com/wiki/Data_storage_options"
Personal tools
Namespaces
Variants
Actions
Navigation
Toolbox