OpenCyc.org HomepageThe CYC® Agenda and CYC® Transcripts

E-Mail Comments to: opencyc-doc@cyc.com
Last Update: 03/28/2002
Copyright© 1996-2002 Cycorp. All rights reserved.

Return to Table of Contents

Managing KB Changes in a Multiple-Image Environment

NOTE: Most of this document does not apply to OpenCyc images, but the document is included to illustrate one way that Cyc (in 2002) handles a multiple-image environment.


Contents:


CYC® Transcripts

If every CYC® image has a separate copy of the CYC® Knowledge Base (KB), how are the different copies of the KB kept in sync with each other?

This is accomplished by having each image write operations to, and read operations from, one or more transcript files.

Transcript File Names

For each CYC® image (each copy of the CYC® KB), there are two transcript files it uses:

Master Transcript

The master transcript file is a complete record of all the operations (changes to the KB) which, when "run" or evaluated, will result in the next (newest) canonical version of the KB. When a CYC® image is started, its copy of the KB is brought into sync with those of other running images (assuming they all have the same version of the KB) by loading the operations in the master transcript file for that KB version.

Local Transcript

The local transcript file is a record of the operations which have been run by a CYC® image. In the course of a knowledge editing session a new local transcript may be started several times.

Communication Settings

There are two parts to the communication status:

What do you want to do with the operations you generate?

Send immediately to the master transcript - local operations are sent to the master transcript as they are processed. Local operations that are sent to the master transcript are also sent to the local transcript.

Store for later - local operations are sent to the local transcript and then stored for possibly sending to the master transcript later.

Do not record - local operations are processed and are NOT recorded. Your image will show the effects of their outcome but you will not have a record of them.

Do you want to receive operations from others?

Yes - You want to process the operations that others are sending to the master transcript.

No - You do NOT want to process the operations from the master transcript.

Therefore, there are six possible combinations for the communication status:

- Sending and Receiving
- Sending
- Storing and Receiving
- Storing
- Not Recording but Receiving
- Not Recording or Receiving

The Communication Status page is where you can view and change your communication settings. When Storing operations, a new section of the communication status page will appear. In this section you are given the option of sending your stored operations or clearing them from storage. Selecting either option will cause a new local transcript to be started for the image. Selecting clear does NOT mean the stored operations will be erased(they will still be in the local transcript).

Queues and the Functional Interface

State-controlling variables that experienced users of CYC® should know about are *use-local-queue?* and *run-own-operations?*.

The value of *use-local-queue?* determines whether or not operations performed in the local image will be executed immediately (thus blocking the execution of other processes), or pushed onto the local queue to be executed at a time determined by the CYC® Agenda. The possible values for this variable are T and NIL. In most cases (most of the time), *use-local-queue?* will be set to T. This means that every KB-modifying operation performed using any of the top-level Functional Interface (FI) functions (e.g., FI-ASSERT) will automatically be pushed onto *local-queue* to be executed by the CYC® Agenda. Usually this is a good thing, because otherwise the operation would be executed immediately where you typed it in (or where it was evaluated internally through the web browser interface). If it happened to be a long operation, you would be forced to wait before you (or another application) could continue to interact with CYC®. However, you may sometimes want the evaluation of a Functional Interface function to happen immediately. In this case, set the value of *use-local-queue?* to NIL by evaluating

  (CSETQ *use-local-queue?* NIL)

in the Interactor web interface page. If you are typing SubL expressions directly to a read-eval-print loop (extremely rare for doing knowledge entry), use the procedure SAFE-FI.

  (SAFE-FI  )

For example, instead of typing the form

  (FI-ASSERT '(#$isa #$Siegel #$Person) #$BaseKB)

you would type

  (SAFE-FI :assert '(#$isa #$Siegel #$Person) #$BaseKB) 

The value of *run-own-operations?* determines whether or not your local CYC® image will process operations originating from it when reading operations from a transcript file. In most cases this should be set to NIL. However, if an image dies and needs to be restarted (due, e.g., to a power failure or irrecoverable memory error), you will usually want it to evaluate its own operations when it is restarted and loads the master transcript file, so that it will be in sync with the other running images. This is just something to be aware of and watch for -- the right thing should happen automatically, and you should never have to manually set the value of this variable.

What is in a transcript file?

Each expression in a transcript file is a list of four elements, with the following form:

([user] [image-id] [date] [encapsulated op])

[user] = the originating user's name
[image-id] = the ID of the originating CYC® image
[date] = date the operation was added to the transcript file
[encapsulated op] = an encapsulated FI operation

An encapsulated operation is one in which all CYC® constants (e.g., #$isa) are referred to as lists including name-id pairs (e.g., (:HP "isa" 260)) rather than as Cyc-constant data objects.

Loading an Arbitrary Transcript File

Arbitrary transcript files can be loaded in two ways. The easiest way is to click on the [Load Transcript File] link in the CYC® Navigator, and follow the instructions presented there.

In a read-eval-print loop, you can also call the function LOAD-TRANSCRIPT-FILE with the appropriate arguments. These arguments include the pathname of the file to be loaded, a boolean value (T or NIL) indicating whether or not operations that originated in the local image should be loaded, and a keyword indicating onto which queue the loaded operations should be pushed. The possibilities for the last argument (the queue keyword) are :local, :remote, and :none. :None means that the Functional Interface commands in the transcript will be evaluated immediately, and other processes will be forced to wait until they are finished.

Calling FI Functions Directly

Direct calls to FI functions act just like editing commands from the CYC® Web Interface: by default, they are queued for execution by the agenda and saved in transcript files. It is important to be aware of this behavior to avoid sharing unintended operations with other CYC® Servers at a site.

When a CYC® image is started and the CYC® init file is loaded, the global variables *use-local-queue?* and *use-transcript?* are initialized to T. With these settings, every FI command you evaluate (even including fi-ask), will be added to your transcript queue and (if your agenda is running) saved to a transcript file. If you are Sending operations or Storing and you send your stored operations, the operations will be appended to the master transcript file, and everyone who is reading from the master file will run your operations.

Changing your communication status is one way to safely alter the setting of these variables. For example, if you go into a Not Recording mode then no operations performed in your CYC® image will be added to its transcript queue. In rare cases, a user might want to "manually" set *use-transcript?* to NIL. If you think you need to manually set these variables, check with your contact at Cycorp. Here is a way to handle a common situation -- you want to run some cleanup operations that will be included in the next KB dump, but you don't want everyone to be forced to run the cleanup:

Cycorp has adopted the informal policy of using one CYC® image as a dedicated KB cleanup and dump image. If you want to run large numbers of cleanup operations to be included in the next dump, you can run them in the dedicated image with the communication status set to Storing and then clear the operations from storage. The local transcript file containing the operations can then be loaded on the dump image (make sure not to send them to the master transcript) immediately before the next dump.

If in the context of some cleanup you write code or evaluate FI commands "manually" (in a SubL read-eval-print loop), use the generalized "safe" dispatcher function for FI commands called SAFE-FI. Within calls to SAFE-FI, *use-local-queue?* and *use-transcript?* are bound to NIL, so there is no danger of unintended operations ending up in a transcript file. For example, instead of calling:

  (FI-ASK '(#$arity #$customers ?Y) #$BaseKB)

you could call:

  (SAFE-FI :ask '(#$arity #$customers ?Y) #$BaseKB)

Note that the first expression above would not return an answer unless *use-local-queue?* were set to NIL. Instead, the result of the evaluation would be pushed onto the local queue, and the evaluated form would return the keyword :queued. However, the second expression is guaranteed to return a result, since *use-local-queue?* is internally bound to NIL.

Additional Comments

Just about everything described above -- transcript file management, setting the values of variables, etc. -- should happen automatically. You should never have to name a local or master transcript file, specify a version number, append files, or directly set a variable's value. If for some reason you believe that you need to do one of these things (or something more arcane), please contact Keith Goolsbey, Nick Siegel, or another member of the Cycorp Technical Board.

What "dump" Means

Readers of this documentation will occasionally encounter the term "dump" and the phrases "do a dump" or "dump the KB". "Dumping the KB" means that one CYC® image will be used to save the current state of its KB out to a set of specially formatted files (.cfasl files). These files will then be used to construct a new version of the KB. In effect, this is like taking a snapshot of the current state of the KB in the dump image and using that snapshot as the starting place for the next image to be built. When a dump has started, the dump image will read no further operations from the master transcript file, and any operations written to that file after the dump has started will not be included in the new image to be built.

Note that users of other copies of the current CYC® image (copies other than the particular image being used to do the dump) can always continue to work (do new knowledge entry) after a dump has started. Just switch to a non-sending communication status. This will start a local transcript file for your new operations. After a new image is built and you have started it, you can then load your transcript file, specifying that the operations should go on the local queue. The operations will then be treated as if they had just now been performed in the new image, and they will be included in the new master transcript file.


The CYC® Agenda

What controls how often, or how frequently, operations are written from the transcript queue to a transcript file? What do all those numbers on the "Server Status" ("System Area") page REALLY mean?

The CYC® Agenda is the process running in a CYC® image which performs all the tasks needed to manage CYC® operations: moving operations from one queue to another, executing operations, reading from transcript files, writing to transcript files, and loading CYC® source code patches.

In computer science terms, the CYC® Agenda is a finite state machine which continually moves from state to state, testing for a certain set of conditions in each state, and causing specific actions to be performed when the conditions hold.

Or you can think of the agenda as a manager who spends all day checking her watch and running back and forth between the offices of her employees, telling each one when it is time to do something.

A Flowchart for the CYC® Agenda

Here is an informal description of the states through which the CYC® Agenda moves, and what it does in each state (leaving out error handling and other technicalities):
(0) Start Agenda
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)

The Server Status page

The [Server Status] web browser page contains a section which looks something like this:

System Area Here are your system settings. Current machine: Lenny Current image number: LENNY-293-19980105094631 Current KB: 111.545 System and patch level: 54.44 Click here to halt this CYC® image.
Queue Information: Master Transcript Operations Read But Unprocessed: 0 Master Transcript Operations Read: 545 Unprocessed Local Queue Operations: 0 Local Queue Operations Processed But Unrecorded: 0 Current Local Transcript Operations Processed And Recorded: 1
Transcript Information: Master Transcript: ./transcripts/0111/cyc-kb-111.ts Characters: 251230 Estimated Ops: 1219 Operations Read: 545 Local Transcript: ./transcripts/0111/LENNY-293-19980105094631-local-0.ts Operations: 1
Historical Information: Remote Operations Processed: 545 Local Operations Processed: 1 Local Operations Sent To Master Transcript: 0
Copyright © 1995, 1996, 1997 Cycorp. All rights reserved.
Here is a line-by-line explanation of the various bits of information displayed in this section:

Current machine: [machine-name]

Indicates the machine that CYC® is running on

Current image number: [CYC® image id]

Indicates the unique id of the CYC® image currently running. A CYC® image id consists of three elements: a machine name, a process ID integer, and a large integer indicating the universal time at which the image was started. (Thus, every image started from a CYC® executable will have a different, unique ID, even if two or more images are started on the same machine).

System and patch level: [patch level]

Indicates the version number (system number and patch level) of the CYC® image.

Master Transcript Operations Read But Unprocessed: [n]

The integer [n] indicates the number of operations that have been read from the master transcript but have not yet been processed.

Master Transcript Operations Read: [n]

The integer [n] indicates the number of operations that have been read and processed from the master transcript.

Unprocessed Local Queue Operations: [n]

The integer [n] indicates the current number of operations sitting in the local queue waiting to be processed.

Local Queue Operations Processed But Unrecorded: [n]

The integer [n] indicates the current number of operations that have been processed from the local queue and pushed into the transcript queue, but have not yet been saved to the local transcript.

Current Local Transcript Operations Processed And Recorded: [n]

The integer [n] indicates the number of local operations that have been processed and saved to the current local transcript.

Master Transcript: [transcript file name]

The name of the master transcript is provided. This file can be examined by clicking on the file name.

Characters: [n]

The integer [n] specifies the number of characters in the master transcript.

Estimated Ops: [n]

The integer [n] specifies the approximate number of operations in the master transcript.

Operations Read: [n]

The integer [n] specifies the number of operations read from the master transcript. This number will be equal to the sum of Master Transcript Operations Read But Unprocessed and Master Transcript Operations Read fro the Queue Information section.

Local Transcript: [transcript file name]

The name of the local transcript is provided. This is the file that all operations generated by this image are being recorded in. This file can be examined by clicking on the file name.

Operations: [n]

The integer [n] specifies the number of operations in the local transcript file listed above.


Go to Top