Monthly Archives: October 2024

The new Scan Repository tool

OpenInsight 10.2.3 brings a much needed update to the venerable Scan Repository tool, providing an updated user interface, full support for new repository types, and some new methods for problem detection and repair. In this post we’ll take a look at each of these areas in more detail, and how the various options work when repairing your system.

A new user interface

The Scan Repository tool is accessed from the IDE main menu via the Tools\Repository menu item and then choosing the “Scan Repository For Errors” option (or alternatively by executing the RTI_IDE_SCAN_REP form via the System Monitor). Users of previous incarnations of the Scan Repository tool will notice an immediate change in the user interface as it is no longer a “wizard” style UI, but now shows all of the scanning options, progress information and results on a single form instead:

The new Scan Repository tool
The Scan Repository Tool

Using the scanner is a simple process:

  • Ensure the Scan Prerequisites are met.
  • Choose the Scan Options.
  • Click the “Analyze” button to begin the scan. Progress information will be shown at the top of the form and the scan can be cancelled at any point.
  • Review the results and, if necessary, select the appropriate repair action. These appear in dropdown lists in the “Action” column of the results table and can be one or more of the following options (depending on the error):
    • Add
    • Delete
    • Ignore
  • Make sure that the items you want to fix are highlighted (selected) in the list (Remember to use the context menu to help with the selection process!).
  • When ready click the “Process” button to begin the repair process.
  • Review the repair results.
    • Note that some repairs may not be able to be completed automatically. In this case the corresponding action item may change to an “Edit” button so that the problem item can be fixed manually.
    • Details of errors encountered during the repair process are shown in a tooltip when the mouse is hovered over a problem item.

Scan Prerequisites

Repairing errors in your repository involves possible changes to some fundamental system components, so the following requirements must be met before the scan can take place:

  • Ensure that you have backed up your system and check the “A recent backup is available” check box.
  • Ensure that all others users are logged out of your system, otherwise the scan cannot proceed. The details of other logged-in users will appear in the Users list to help identify them.

Scan Options

There are two categories of scanning options. The first, “Types and Classes”, scans the core repository records to ensure that they are all correct, while the second category, “Entities” scans the normal application components.

Scanning for Types and Classes

The functionality of the repository is based heavily on the information contained in the following three locations, so it is essential that these items are correct:

  • REPFAMILY entities (SYSREPOSTYPEFAMILIES records)
  • REPTYPE entities (SYSREPOSTYPES records)
  • REPCLASS entities (SYSREPOSCLASSES records)

These are referred to as FTC entities (Family/Type/Class). They can be one of two types:

  1. “Known” – these are the FTC entities are that are supplied by Revelation.
  2. “Custom” – these are non-Revelation FTC entities added by developers that have been marked as valid in a previous “Types and Classes” scan (When an “unknown” FTC entity is first encountered in a repository scan it can be flagged as “custom” so it is considered valid in subsequent scans – see the “Unknown” class of errors in the table below).

Verify Repository Types and Classes

The state of the FTC entities can be validated by checking the “Verify Repository Types and Classes” box which ensures that this process takes place before any other normal entities are checked.

Checks performed during this scan include:

  • All known FTC entities are present and not missing any critical information.
  • All custom FTC entities which have been flagged as valid are not missing any critical information.
  • All other FTC entities that are not flagged as custom are reported as “Unknown” so they can be reviewed.

The Verify Repository Types and Classes scan can produce the following errors:

ErrorDESCRIPTION
Missing Known Type Family,
Missing Known Type,
Missing Known Class
One of the known FTC entities is missing. The default version can be added back into the system using the “Add” action.
Unknown Type Family,
Unknown Type,
Unknown Class
An unknown FTC entity has been found. It can be added as a custom type using the “Add” action, or removed using the “Delete” action. Items that have been added are marked as custom and treated as a valid FTC type in future scans.
Missing Repository RecordThe raw FTC body record is missing a corresponding SYSREPOS header record. A default SYSREPOS record can be created using the “Add” action.

(Note: OpenInsight repository entities are usually comprised of two components: A “header” record in the SYSREPOS table, and a “body”. The latter can be a record in a table or an OS file depending on the type of entity).
Bad Repository Sub-keyThe SYSREPOS header record for the FTC entity has an invalid SUBKEY entry. The “Add” action may be used to correct this.
Missing Title,
Missing Icon,
Missing Image,
Missing Type Family,
Missing SCM Format,
Missing SCM Include,
Missing Storage Type,
Missing Location,
Missing RDK Extract Type,
Missing Key Format,
Missing Designer Tool
The FTC body record is missing some required information.

If this is a known FTC entity then the “Add” action may be used to reset the missing value to its default setting.

If this is a custom FTC entity the “Add” action will switch to an “Edit” button, which opens the entity for editing in the IDE.
Missing Event ClassA custom event (added via the Event Designer tool) is missing a corresponding SYSEPOSCLASSES record. The “Add” action may be used to add a default record.

Reset to default values

Selecting this option ensures that all known Revelation-supplied FTC entities are reset to their default values during the scan. This option can be used to ensure that your repository is always in a good state.

Scanning Entities

This part of the process scans the rest of the repository based on the options chosen:

OptionDescrIPTION
Scan for missing entity headersScans a set of core tables looking for missing repository header (SYSREPOS) records.

Due to the fact that it can allow “lost” entities to be readded to the repository it is recommended that this option be run before any others.

If this option is selected all of the other Entity scanning options will be disabled.
Scan repository keys onlyOnly checks the validity of repository header (SYSREPOS) keys and does not check the state of the entity itself.

The only other option allowed with this selection is “Use strict validation criteria for entity IDs”.
Validate “uses” dependenciesEnsures any entities flagged as “used” by the entity being scanned actually exist. If they don’t the link is automatically removed.
Validate “used by” dependenciesEnsures any entities flagged as “used-by” by the entity being scanned actually exist. If they don’t the link is automatically removed.
Validate “documentation” dependenciesEnsures any documentation entities linked to the entity being scanned actually exist. If they don’t the link is automatically removed.
Scan for orphaned event handlersEvent entities (source and executable) are checked to ensure that they are linked to their parent forms, and that the parent forms actually exist.
Scan for missing entity bodiesValidates that each repository entity has a corresponding body record or OS file as appropriate.
Use strict validation criteria for entity IDsValidates that keys in the SYSREPOS table conform to a limited set of characters.

When the “Scan for missing entity headers” box is checked the Entities scan can produce the following errors:

ErrorDescription
Orphan – Missing SYSREPOS [<id>]An entity body record is missing a corresponding SYSREPOS header record. The “Add” action may be used to add a default header record, the “Delete” action may be used to delete the orphaned body.
Orphan – Bad App [<id>]An entity body record is linked to a non-existent application ID. The “Delete” action may be used to delete the record (and any associated SYSREPOS header record).
Orphan – Suspected Bad App [<id>]Due to the polymorphic nature of the records in some tables (primarily SYSPROCS and SYSOBJ) it is not always possible to accurately determine what type of record is being checked.

This error can be raised in such circumstances, and the “Delete” action may be used to remove the record in question.

However, it is advised that you review the record in question before you delete it!
Orphan – Bad Key Format [<id>]A known type has a badly formatted key. The “Delete” action may be used to remove the record in question.

When the “Scan for missing entity headers” box is not checked the Entities scan can produce the following errors:

ERRORDescription
Bad KeyThe entity has a badly formatted SYSREPOS key. The “Delete” action may be used to remove the entity.
Bad AppThe APPID of the SYSREPOS key refers to an invalid application ID. The “Delete” action may be used to remove the entity.
Bad TypeThe TYPEID of the SYSREPOS key refers to an invalid SYSREPOSTYPES type ID. The “Delete” action may be used to remove the entity.
Bad ClassThe CLASSID of the SYSREPOS key refers to an invalid SYSREPOSCLASSES class ID. The “Delete” action may be used to remove the entity.
Deprecated EventAn entity has been found for a class of event that is no longer supported in version 10. The “Delete” action may be used to remove the event in question.
Strict TestThe SYSREPOS header key failed the “strict” test and is considered to be badly formatted. The “Delete” action may be used to remove the entity.
No locationThe SYSREPOS header record is missing a required location (e.g. an IMAGE type is missing the location of the OS image file). The “Delete” action may be used to remove the entity.
No BodyThe entity is missing a body record or OS File as determined by its type. This can happen if the body record exists but is null, or the ACCESS method returns null. The “Delete” action may be used to remove the SYSREPOS header record.
No Body <FSError>The entity’s body record cannot be accessed because of a filing system error. The “Delete” action may be used to remove the SYSREPOS header record.
Open ErrorA table open error occurred when checking a DBTABLE type entity. The “Delete” action may be used to remove the SYSREPOS header record.
Orphan – Missing FormAn event script entity is referring to a form entity that does not exist (i.e. there is no SYSREPOS header record for the form). The “Delete” action may be used to remove the event entity.
Orphan – Missing Form BodyAn event script entity is referring to a form entity whose body record does not exist (i.e. there is no SYSREPOSWINS or SYSREPOSWINEXES record for the form). The “Delete” action may be used to remove the event entity.
Orphan – Missing ControlAn event script entity is linked to a control that does not exist on the form. The “Delete” action may be used to remove the event entity.
Orphan – Event Not LinkedAn event script entity has been found for a control or form that does exist but it is not linked to it. The “Add” option may be used to create the link, or the “Delete” action may be used to remove the event.

(Note: Please review the control and event in question before you decide to add it as there may be a good reason that it is no longer linked!)
Orphan – Missing Script ExeAn event script debug-table entity has no corresponding event script executable entity. The “Delete” action may be used to remove the event script debug-table entity.
Orphan – Event Cannot Be LinkedAn executable event script entity cannot be dereferenced to an owner. The “Delete” action may be used to remove the executable event script.

The Scan Repository context menu

The results list of the Scan Repository tool includes a comprehensive context menu to help with selecting the items to repair along with an export option and an easy way to view the details of a specific entity:

Menu ItemDescription
Process all itemsExecutes the specified repair action (if appropriate) for all entities in the results list.
Process selected items onlyExecutes the specified repair action (if appropriate) for all selected entities in the results list.
Set all items to “Add”Sets the action option of all items in the results list to “Add” (but only if they have an “Add” option).
Set all items to “Delete”Sets the action option of all items in the results list to “Delete” (but only if they have a “Delete” option).
Set all items to “Ignore”Sets the action option of all items in the results list to “Ignore” (but only if they have an “Ignore” option).
Set all selected items to “Add”Sets the action option of all selected items in the results list to “Add” (but only if they have an “Add” option).
Set all selected items to “Delete”Sets the action option of all selected items in the results list to “Delete” (but only if they have a “Delete” option).
Set all selected items to “Ignore”Sets the action option of all selected items in the results list to “Ignore” (but only if they have an “Ignore” option).
Select all itemsSelects all of the items in the results list.
Clear all selectionsUnselects all of the items in the results list.
View <entity>Attempts to open both the SYSREPOS header record and body record (if possible) for the specified entity. These are opened as “raw” records in the IDE.
Export resultsExport the contents of the results list to a CSV file.

Conclusion

The new Scan Repository tool is available in OpenInsight v10.2.3 onwards. Hopefully it will allow you to keep your repository in good shape moving forwards!

Directory Management in OpenInsight 10

Over the years there have been several different and disparate ways of managing directories in OpenInsight, and not all of them fully documented. In this post we’re going to take look at the “official” preferred methods, along with a mention of the deprecated ones too.

Preferred methods

  • The FILESYSTEM object (for Event Context)
  • RTI_OS_Dir stored procedure (for non-Event Context)

Deprecated methods

  • Utility stored procedure
  • RTI_OS_Directory stored procedure
  • DirExists stored procedure
  • MkDir stored procedure
  • UtilityMakeDir stored procedure
  • UtilityRemoveDir stored procedure
  • UtilityRename stored procedure

A note on directory management and context

As you may be aware, an OpenInsight application runs in one of the following contexts:

  • Event Context – This applies when your Basic+ programs are called in response to an event from a standard OpenInsight application form or control (i.e. an application managed by the Presentation Server).
  • Non-Event Context – This applies to applications that run Basic+ programs outside of the Presentation Server using the RevCAPI interface to manage an instance of RevEngine. These are usually “Inet” or O4W web applications, but also include other methods like the RevRun.exe program too.

Therefore, one of the most fundamental considerations when choosing which directory management method to use is the context in which it is called: As a rule, when running in Event Context, you should always prefer to use the FILESYSTEM object for directory management, otherwise you should choose the RTI_OS_Dir stored procedure instead.

Note that if there is a possibility that your Basic+ programs will be executed in different contexts at runtime (i.e. you share them between contexts) then you should invoke the IsEventContext stored procedure to determine which method to use.

For example, here is a simple context-aware code snippet that removes a directory:

   Declare Function IsEventContext, RTI_OS_Dir, Exec_Method
   $Insert PS_FileSystem_Equates
   $Insert RTI_SSP_Equates
   
   ErrText = ""

   If IsEventContext() Then
      // Use the FILESYSTEM object
      If Exec_Method( "FILESYSTEM", "REMOVEDIR", DirName ) Else
         ErrInfo = Get_Property( "FILESYSTEM", "FILEOPRESULT" )
         ErrCode = ErrInfo<PS_FOR_ERRORCODE$>
         ErrText = ErrInfo<PS_FOR_ERRORTEXT$>
      End
   End Else
      // Use RTI_OS_DIR
      Call Set_Status( SETSTAT_OK$ )
      If RTI_OS_Dir( "REMOVEDIR", DirName ) Else
         Call Get_Status( ErrText )
      End
   End

Using the FILESYSTEM object

The FILESYSTEM object supports the following methods to manage directories. It integrates fully with the Windows Shell and provide the best user experience in Event Context:

  • COPYDIR
  • DIREXISTS
  • GETSPECIALDIR
  • MAKEDIR
  • MOVEDIR
  • REMOVEDIR
  • RENAMEDIR

The FILESYSTEM object is fully documented here.

Using the RTI_OS_Dir stored procedure

This stored procedure was added to OpenInsight 10 to provide a non-Event Context version of the functionality exposed by the FILESYSTEM object. It supports the following methods:

  • COPYDIR
  • DIREXISTS
  • GETTEMPDIR
  • MAKEDIR
  • MOVEDIR
  • REMOVEDIR
  • RENAMEDIR

The RTI_OS_Dir stored procedure is fully documented here.

(Note that there is no direct equivalent of the FILESYSTEM GETSPECIALDIR method, due to the fact that it is very Windows-specific – the GET_SPECIAL_FOLDER stored procedure should be used instead.)

The Utility stored procedure (Deprecated)

As long-time readers of this blog will know, this stored procedure was deprecated some years ago at the outset of the OpenInsight 10 project, and it is now basically a thin wrapper around several SYSTEM and FILESYSTEM object methods. It supports the following directory management methods:

  • MAKEDIR
  • REMOVEDIR
  • RENAMEDIR

When called in Event Context each of these methods forwards the request to the FILESYSTEM object. When called outside of Event Context these methods call the following stored procedures instead:

  • UtilityMakeDir
  • UtilityRemoveDir
  • UtilityRename

See below for more details on these.

The RTI_OS_Directory stored procedure (Deprecated)

This stored procedure was deprecated in favor of RTI_OS_Dir as it is very platform specific and makes use internally of RTI_OS_Dir, Utility, and an OLE interface to the Windows Shell, making it less performant than calling those procedures directly. Like Utility it is still supported but will not be updated further with any new functionality.

DirExists stored procedure (Deprecated)

This stored procedure is a simple wrapper around the RTI_OS_Dir DIREXISTS method and so has been deprecated, but can still be used by existing code.

MkDir stored procedure (Deprecated)

This venerable stored procedure is a DLL function that calls the Windows API CreateDirectory function. It has been deprecated in favor of the RTI_OS_Dir MAKEDIR method but can still be used by existing code.

UtilityMakeDir, UtilityRemoveDir and UtilityRename stored procedures (Deprecated)

These three functions form part of the original internals of the Utility stored procedure, and use a “raw” C interface that requires explicit null-terminated strings. They are all still available to use, but are deprecated in favor of the RTI_OS_Dir MAKEDIR, REMOVEDIR and RENAMEDIR methods.

Conclusion

As you can see, version 10 has pulled together the many different historical methods of directory management and consolidated them into two entities that can be used dependent on the execution context. These should be the preferred methods of directory management in your OpenInsight applications.

(As of the time of writing the full documentation for RTI_OS_DIR and GET_SPECIAL_FOLDER is in preparation and will be available shortly. All methods described here apply to version 10.2.3 and later.)

(EDIT: Full documentation for RTI_OS_DIR and GET_SPECIAL_FOLDER is now live on the Revelation Wiki.)