Local Repository Cache

From HEWIKI
Jump to: navigation, search

Contents

The Local Repository Cache (LRC) is the mechanism on the local computer which holds a cache of some of the objects and files available in the Repository, downloaded from the master database.

Interaction with clients

Infrastructure

In terms of the LRC's place in the Engine infrastructure, it works like this: Whenever the FireStorm API is started up, a worker thread is created on the client, whose primary duty is to accomplish the time-expensive portions of storing and retrieving resources to disk, while the main thread continues to display the game on the client. When a client (such as HeroBlade or the Player Client) identifies that it needs an asset from the Repository, this communication is done via the remote Repository Server process, which is what communicates with the actual Repository database. When the art is downloaded from the Repository to the Local Repository Cache, it is stored on local disk, and then announced as delivered, to the client that requested the resource.

To observe this process in action, open the Repository Requests panel to see current outstanding requests. Those that appear and disappear rapidly from the panel are probably being filled from the LRC, while those that stay on the panel for longer are probably waiting for response from the Repository Server.

File size

As a developer builds the game, more and more art will be stored in their Local Repository Cache, which could cause it to become quite large (hundreds of megabytes). There is not currently any mechanism to automatically delete unused or out-of-date items, so in some circumstances, such as to save disk space or possibly troubleshoot some corrupted art, a developer may wish to purge the Local Repository Cache. However, be aware that doing this will purge all art from the cache, and require re-downloading of all art assets.

Location

The LRC location is determined by your Operating system. Windows 7 specifies where program data should be stored according to the person who is currently logged in. In Windows 7 this is commonly the c:\Users\<user name>\ directory. The client configuration CacheFilePath, which by default is set to \HeroEngine\Repository\, dictates the path beneath the location specified by the OS.

The LRC's location can vary based on those configuration values, but typically can be found (depending on the Operating System) in:

Windows Vista/7 (Example path, the actual path may vary)
C:\Users\<user name>\AppData\Local\HeroEngine\RepositoryCache

Depending on when your HeroCloud world was setup your path may look something more like:
C:\ProgramData\"Your World Name"\Repository

There are valid reasons to have repositories for the different worlds of you project to be found in different locations so that they have unique LRCs. The most common reason to do this would be to separate worlds that can (and should) share a common repository of assets such as production worlds from worlds that ideally should be separate in case potential issues arising from development.

Client Installation and the LRC

There are essentially three different ways to handle the client installation for production:

The production client by default falls in the category of streaming install and consequently functions identically to HeroBlade when the LRC is empty. The remaining types of installations require the creation of a primed cache that is then included in the client installation.

The primed cache for a full installation is generally generated utilizing the RepositoryBrowser (see below Populating the local cache). The disadvantage of a priming the cache in this manner is that you may include content that is not actually needed or used in the content available to your customers.

Guided repository cache population is accomplished in one of two different ways, both of which allow you to use your knowledge of how your game is played to choose exactly what goes into the installer. This helps ensure the installation is as small as possible while having a better initial user experience than purely streaming installs.

Basic steps for a Guided Repository Cache that contains all Game Content:

Contents

The folders are organized by Game ID, which should be unique across worlds (if not, this can cause GUID conflicts). These numbered folders contain specific worlds' filename/guid/content mapping information. The content mapping is a file #, row # link to the _assets folder (a specific denormalization from the hash-size key due to measured performance issues).

The _assets folder contains the actual content-addressable file data. Content-addressing is built from a file-size file-hash pair, which results in a 20-byte binary key. Assets that share the same content-addressing information are coalesced in the assets databases.

Populating the local cache

The "Prime Local Cache" checkbox in the "Configure Settings" of the Repository Browser

It is possible to make a copy of a local cache (called a "Fully Populated Cache" or a "Primed" local cache). This is a cache that has been initialized on your local system, and which may then be copied to other computers. This saves time so that no longer does each client need to painstakingly download all of the files needed. The pre-generated local cache will then have all of the resources available from the Repository, and could then be packaged up and used in the install process to initialize the LRC of other clients.

To fully populate the local cache:

PrimeLocalCache2.png

To copy this file to another client, simply copy two folders:

Shared files

If a developer accesses multiple worlds, they may need multiple clients, but all worlds will share the same Local Repository Cache. The only differences will be fairly small, such as a separate .map file stored in each Local Repository, for each world.

For shared repository data on the server side, see Sharing repositories.

Removing LRC files

Removing the _assets folder REQUIRES removal of all of the world-specific folders.

Removing a world-specific folder does not require removing the _assets folder, as the local repository cache code will "re-attach" the same content to the entry already in the assets folder.

As the underlying database is a straight port with one patch of SQLite, tools such as "SQLiteSpy" can browse the contents of the Local Repository Cache database files. The corresponding code in FireStorm is isolated to $/firestorm/internalcode/clientrepository/ LRRepositoryMapStorage.cpp and LRBlobStorage.cpp, coordinated by LRStorage.cpp, with utilities in LRStorageUtils.cpp.

To purge the LRC, see Purging the Local Repository Cache

Repository Corruption

The LRC can only become corrupt (aside from user action, such as doctoring the data in place) whenever there is a sudden shutdown of the system without time for the operating system to write all pending data to disk. IE: a power failure, or a blue-screen of death.

The solution is to "move aside" the LRC, and allow the client PC to recreate the empty LRC and re-download all of the necessary assets. Obviously, the first time entering will take longer than usual, and may discover errors in the client-side script code that made assumptions about asynchronicity and timing of data being available. Typically, the user then restarts HeroBlade and "it just works".

See also

Personal tools
Namespaces
Variants
Actions
Navigation
Toolbox