Managing servers and processes
Servers and processes and managed via the Master Control Console client, which communicates with the Master Control Daemon on the server. It displays the structure of the component tree, which it gets from the Master Control Database, along with the status of those components, which it gets from the Master Control Daemon. It also allows you to view logging information from the various processes, and control the various servers, such as to open and close them to new processes, or to stop processes entirely.
Starting up a server
- First, make sure the server is running (meaning that its name is displayed in a green color) by looking on the Servers branch of the tree view of the Master Control.
- If it is red, that means the
fireupdaemon is not running on that machine. In this case, it will be necessary to VNC to it, and start it manually.
- If it is red, that means the
- Click on the universe (such as HE_TRAINING). In the status window there will be a list of all of the processes in that universe. If any processes are red, click the
Fireupbutton, and it will start up each of those processes in order.
Determining whether or not a server is up
- If all of a server's processes show up as green, and it has dynamic processes running, it's up.
- Look at the Master Control's "reported statistics." It should display each world and whether or not it is up.
Note: Processes can go into a "bad" state, meaning that a process may look like it's up, but may be halted after detecting a non-recoverable error. This can be detected by looking at the process's statistics, which should have a label such as, "STATE: BAD". If this happens, then:
- Look at the logs to determine what put it into the state
- Fix the problem
- Shut down the process (see "shutting down" section below)
- Restart the process (see "starting up" above)
Determining if the Service Directory Process is up
Each Service Directory process should have two other supporting processes:
If all three processes do not exist, or one of the processes shows up as being in a bad state, shut it down and restart it (see above section).
Shutting down a server
- Select what you want to shutdown, for example the HE_TRAINING universe.
- Click SHUTDOWN. The Master Control will then shut down each process, in order.
If a process fails to shutdown within the allotted time, an error message will show up. The message will state that the process didn't shut down, and that the automated shutdown process was stopped. In most cases this is just because some of the processes took longer to shutdown than others, and timed out. The best practice at this point is simply to wait for a moment, and then try again.
Terminate a server
If a process is flashing, that means something bad happened to it, like a crash, or it shut itself down because it sensed a problem of some sort. To learn more, expand the tree to view that process and select it by clicking on it. This will then display information about that process in the Status Window.
- Look at the "STATUS" of the process. Also look at its ExitCode. This will either be
STILL_ACTIVE, or, if the process is no longer running, it will be a number.
A process that is flashing red or hung firing up/shutting down will need to be dealt with before it can be started again. If necessary, select the process and select TERMINATE.
Issuing the terminate command will order the system to immediately kill the specified process without giving it a chance to cleanly shutdown. It will then switch to the shutdown state.
Listening to log messages
Through the Master Control, you can listen to current log messages from any process, and view the history of log messages from any process, going back 500 log messages per process.
- Right-click on a process, to watch just that single process.
- Right-click on a world or service, and choose >Watch > Recursive to watch everything in that world or service.
- Once watching a group of processes, you can selectively ignore different ones, to focus on just the ones that you are interested in.
While you are watching, log messages will show up in the Logging window below the Component Tree.
Through the Master Control Console, it is possible to view the performance of any process or server, such as its CPU load. Whichever servers are currently visible in the Component Tree, will be displayed in the CPU Load window.
Selecting a single server or process will cause that server or process's information to be displayed in the Status Window. Various tabs within the window provide additional information:
Not currently implemented
Similar to the Windows Task Manager, this displays dynamic and static processes.
- Static processes are defined in the Master Control Database. They are started automatically by the server, and are listed as subprocesses of the World in the Component Tree.
- Dynamic processes are run "on the fly" by HSL. They can be viewed by opening up the appropriate server in the Component Tree, but are not associated with a specific world.
To stop a process, locate it in the Component Tree, select it, and choose the SHUTDOWN option in the top menu.
The Chat Window
Typing a message into the Master Control's chat window input box will send a message to anyone else who has logged into that universe's Master Control.
Chat Window Commands
Other commands available in the chat window are:
/RESET- Will load new configuration values from the Master Control database into the Master Control daemon
/SHUTDOWN- Shuts down the Master Control and restarts it. This will not affect the worlds -- only the Control Console
/HELP-- displays this list of commands
/SERVER-- Shows the state of servers, whether they are OPEN or CLOSED. If a server (also called a host) is open, then it is available to start up dynamic processes, depending on which server group that it is assigned to. If it is closed, no dynamic processes will start on it.
/SERVER CLOSE <servername>- closes a server. This will not stop existing processes!
/SERVER OPEN <servername>- opens a server, making it available to run new processes
/LISTEN- This is the same as right-clicking on a universe, service, or process in the component tree, to listen to log messages from those processes
/LISTEN <process id>
- To find a process ID, click on that process in the Component Tree, and its ID will appear in the Status Window
The server logs are located at:
The file path may change depending on name, which will usually be the name or an abbreviation of your own company.