Category Archives: Basic+

OpenInsight 10.3 Update

As work continues on OpenInsight 11 we’re still working on OpenInsight 10 to ensure that we’re fixing bugs and implementing a few new features wherever we can (and of course folding them into v11 as well).

OpenInsight 10.3 is the next release and about to hit Beta status, and we’ll announce the release date in due course. Originally we planned to release a smaller 10.2.5 version with just the bug fixes, but as we added a major component update and a few other new things, we decided to bump it up a little, so here’s what you’ll be getting:

An updated OIPI “Classic” (OCX) version

As you know, we supply two main printer engines in OpenInsight: One is based on OLE (OCX) technology (aka. Get/Set_Printer1) , and the other is based on .NET technology (aka. Get/Set_Printer2). The reason for adding the latter version to OpenInsight was mainly because the OLE version didn’t support Unicode at all, and this was becoming a vital requirement for modern applications. The big downside of the .NET version, however, was the speed, and it was generally slower than the OLE version, which was painfully noticeable on some long reports and print jobs.

The good news is that with version 10.3 we’ve implemented the following in the OIPI OCX version:

  • Full Unicode support in the document object and PDF exporter.
  • Support for compression, passwords and encryption in the PDF exporter.
  • Improved the UI feedback in the PDF Exporter
  • Vastly improved the speed of the PDF exporter (we had some reports go from 12 ppm to 600 ppm!)
  • Added text-selection functionality in the Print Preview window to match the .NET version.

So hopefully with that you’ll get the best of both worlds – Unicode and speed!

Added the “Find” API to the WEBVIEW control

Microsoft finally added a Find API in the WEBVIEW control so we’ve now exposed that too. This means that you’ll be able to programmatically control the “Find” functionality in your WEBVIEW controls (You can even use your own Find dialog if you prefer closer integration with your normal OpenInsight GUI forms instead).

Added support for multiple-selection listboxes and drag-reordering

LISTBOX controls have always supported a simplified Drag and Drop ability via the ALLOWDRAGREORDER property, but it was always limited to single-select mode. Due to popular demand we’ve updated this to include multiple-selection mode too (And for those of you wondering, we’re hoping to get full OLE Drag and Drop finished for version 11, honest… ).

Of course there are several other bug fixes and performance improvements we’ve added as well, so be sure to check out the Beta release when it goes live!

Dynamic arrays and offset caching

As you know, dynamic arrays in OpenInsight are internally represented as delimiter-separated strings commonly containing fields, values and sub-values. Traditionally, locating an element required scanning the string from the start to find the required delimiters at the specified index . This can be a performance issue on large arrays, so one of the new features we’ve added to OpenInsight 11 is index/offset caching.

This is a technique that involves caching the byte offsets of previously located field, value and sub-value indexes when you use the “<>” operators, and the Insert/Extract/Replace/Delete functions. This means that we don’t always have to start scanning from the beginning of the string each time an element is accessed.

For example, suppose field 67 is located at byte offset 234, and field 68 begins at offset 245.

  • Without caching, locating field 68 requires rescanning the dynamic array from the beginning to find all preceding field marks.
  • With caching, once field 67 has already been located, the search for field 68 can begin from offset 234 instead of offset 1.

This optimization is especially effective when forward-iterating through large dynamic arrays, where successive accesses are close together. In previous versions, repeatedly accessing successive fields could result in near O(n²) scanning behaviour for large arrays. Offset caching eliminates much of this repeated work.

Consider the following program which processes 10000 SYSOBJ keys:

compile function test_dyn_cache( void )
$insert rti_Struct_Equates
$insert logical
select @file_sysobj
eof = FALSE$
idList = ""
loop
readNext id else eof = TRUE$
until eof
idList := id : @fm
repeat
idList[-1,1] = ""
idList = field( idList, @fm, 1, 10000 )
ctr = fieldCount( idList, @fm )
pf = blank_Struct( "MSWIN_LARGE_INTEGER" )
pcStart = blank_Struct( "MSWIN_LARGE_INTEGER" )
pcEnd = blank_Struct( "MSWIN_LARGE_INTEGER" )
call msWin_QueryPerformanceFrequency( pf )
call msWin_QueryPerformanceCounter( pcStart )
for x = 1 to ctr
id = idList<x>
next
call msWin_QueryPerformanceCounter( pcEnd )
pcStart = struct_To_Var( pcStart, "MSWIN_LARGE_INTEGER" )
pcEnd = struct_To_Var( pcEnd, "MSWIN_LARGE_INTEGER" )
pf = struct_To_Var( pf, "MSWIN_LARGE_INTEGER" )
pcTime = ( ( pcEnd - pcStart ) * 1000 ) / pf
call send_Dyn( "TEST_DYN_CACHE processed " : ctr : " items -> " : pcTime : " ms" )
return ""

Running this benchmark on the same machine using v10.2.4 took approximately 360ms, while OpenInsight 11 completed the same test in around 1–2ms.

For small dynamic arrays the difference may not be noticeable, but performance improvements become increasingly significant as array sizes grow.

This means you can continue using the familiar “<>” syntax for sequential dynamic array processing without needing to rewrite code using LOOP/REMOVE or LOOP/[] parsing patterns.

Introducing OpenInsight 11

With the recent release of OI 10.2.4 it’s time to look ahead at OpenInsight 11 and some of the new features that will be heading your way with the next major release. We’ve been working on it for some time, with a focus on cloud-based features, and there’s plenty to talk about over the next few months as we get it ready, but in the meantime here’s a few highlights:

Revelation Application Server

This is a new native Windows Service application that can host one or more concurrent OpenInsight application instances in a variety of modes:

  • Background Task processing
  • Index Server processing
  • General-purpose “engine-server” with access via a Basic+/Windows DLL/.Net class library/Linux SO, Python wrapper etc.
  • Thin-Client mode for RTP57R (see below)
  • Web-CGI Server (Windows ISAPI, Linux S/CGI, FastCGI, CGI, PHP modules)
  • Internal Web Server with reverse proxying for use with Linux Ngnix/Apache, HTTPS and static file serving support.
  • MCP endpoint for AI access.

RTP57R – Cloud-Optimized Data Access

With the new Application Server we’ve introduced a specialized filing system driver engineered with the Thin-Client mode to meet the demands of distributed applications. As you know, when databases migrate to remote cloud servers and corporate WANs, connection stability and latency often become greater bottlenecks than raw bandwidth. RTP57R (the “R” stands for Remote) addresses these challenges by seamlessly offloading processing to the server when needed and minimizing “chatter” across the wire.

AI Integration

OpenInsight 11 will also feature AI integration features, including a “chat” interface to agentic AI that can be either self-hosted or available through subscription. The Application Server provides an MCP (Model Context Protocol) interface to expose tools and resources to an AI client – we provide a standard set of tools for stored procedure and general DB access, and you can define your own as well.

Basic+ DYNLIST data type

We’ve added a new data type called a “DynList” to Basic+. Basically this is similar to a dimensioned array but it can be resized at runtime. If you’re used to working with other languages then you can think of this as similar to a C++ vector or a C# List<> type etc. DynList variables can store other DynList variables so you can implement fully multi-dimensional data structures.

Basic+ JSON data type

We’ve also made JSON a first class citizen and implemented a set of new JSON functions into Basic+. OI10 included a set of JSON functions exported from RevJSON.dll, but there was always the issue of manual handle management, and some memory management issues. Having JSON in Basic+ removes these concerns.

And there’s more…

So that’s just a few new things that we can tell you about now, and of course there will be more work on improving the performance and functionality of other existing areas like the IDE, Form Designer and other tools. We’ll reveal more details in future posts.

$DEBUG – the safer alternative to Debug

Probably one of the most annoying issues that developers encounter is distributing programs to customers that contain unintentional ‘Debug’ statements. These are often left in from testing, and, with the pressure of deadlines to deliver, they are very easy to overlook.

In order to help alleviate this issue OpenInsight 10 implements a “$DEBUG” statement that you can use in place of the normal “Debug” statement: The difference here is that the debugger only triggers if the code is running on the same machine that it was compiled on, so if your program does manage to escape with a compiled $DEBUG statement it shouldn’t execute on other machines.

There are two variations of $DEBUG. The first is a simple “$DEBUG” statement like so:

Write Record To hFile, Key Else
// Failed etc...
$DEBUG
End

The second type of $DEBUG statement behaves in the same manner as described above, but also includes the ability to define a simple expression to test, and will only trigger the debugger if the test is met (This is similar to the way that the $ASSERT statement works). For example:

// Only debug if x is equal to 3
$DEBUG( x == 3 )

$DEBUG can be used in any version of OpenInsight 10 with the following caveats:

  • From version 10.2.3 onwards the raw workstation name (e.g. “REVWS07” ) is used to test if the debugger should execute.
  • Prior to this (version 10.2.2 and earlier), the entire contents of @station is used to test. The issue here is that @station usually includes the OpenInsight process ID (e.g. “REVWS07_45383” ) so it will only debug within the instance of OpenInsight that it was compiled on. This can be a little too restrictive unless you are willing to recompile your programs after restarting your application, hence the change for version 10.2.3 mentioned above.

Note that you can also disable $DEBUG statements when compiling, in which case the debug tests are never included in the object code.  To do this simply use the NODEBUG token with the #define statement like so:

#define NODEBUG

So, hopefully you can use the $DEBUG statement for your test debugging and have a smoother experience with your customers!

Directory Management in OpenInsight 10

Over the years there have been several different and disparate ways of managing directories in OpenInsight, and not all of them fully documented. In this post we’re going to take look at the “official” preferred methods, along with a mention of the deprecated ones too.

Preferred methods

  • The FILESYSTEM object (for Event Context)
  • RTI_OS_Dir stored procedure (for non-Event Context)

Deprecated methods

  • Utility stored procedure
  • RTI_OS_Directory stored procedure
  • DirExists stored procedure
  • MkDir stored procedure
  • UtilityMakeDir stored procedure
  • UtilityRemoveDir stored procedure
  • UtilityRename stored procedure

A note on directory management and context

As you may be aware, an OpenInsight application runs in one of the following contexts:

  • Event Context – This applies when your Basic+ programs are called in response to an event from a standard OpenInsight application form or control (i.e. an application managed by the Presentation Server).
  • Non-Event Context – This applies to applications that run Basic+ programs outside of the Presentation Server using the RevCAPI interface to manage an instance of RevEngine. These are usually “Inet” or O4W web applications, but also include other methods like the RevRun.exe program too.

Therefore, one of the most fundamental considerations when choosing which directory management method to use is the context in which it is called: As a rule, when running in Event Context, you should always prefer to use the FILESYSTEM object for directory management, otherwise you should choose the RTI_OS_Dir stored procedure instead.

Note that if there is a possibility that your Basic+ programs will be executed in different contexts at runtime (i.e. you share them between contexts) then you should invoke the IsEventContext stored procedure to determine which method to use.

For example, here is a simple context-aware code snippet that removes a directory:

   Declare Function IsEventContext, RTI_OS_Dir, Exec_Method
   $Insert PS_FileSystem_Equates
   $Insert RTI_SSP_Equates
   
   ErrText = ""

   If IsEventContext() Then
      // Use the FILESYSTEM object
      If Exec_Method( "FILESYSTEM", "REMOVEDIR", DirName ) Else
         ErrInfo = Get_Property( "FILESYSTEM", "FILEOPRESULT" )
         ErrCode = ErrInfo<PS_FOR_ERRORCODE$>
         ErrText = ErrInfo<PS_FOR_ERRORTEXT$>
      End
   End Else
      // Use RTI_OS_DIR
      Call Set_Status( SETSTAT_OK$ )
      If RTI_OS_Dir( "REMOVEDIR", DirName ) Else
         Call Get_Status( ErrText )
      End
   End

Using the FILESYSTEM object

The FILESYSTEM object supports the following methods to manage directories. It integrates fully with the Windows Shell and provide the best user experience in Event Context:

  • COPYDIR
  • DIREXISTS
  • GETSPECIALDIR
  • MAKEDIR
  • MOVEDIR
  • REMOVEDIR
  • RENAMEDIR

The FILESYSTEM object is fully documented here.

Using the RTI_OS_Dir stored procedure

This stored procedure was added to OpenInsight 10 to provide a non-Event Context version of the functionality exposed by the FILESYSTEM object. It supports the following methods:

  • COPYDIR
  • DIREXISTS
  • GETTEMPDIR
  • MAKEDIR
  • MOVEDIR
  • REMOVEDIR
  • RENAMEDIR

The RTI_OS_Dir stored procedure is fully documented here.

(Note that there is no direct equivalent of the FILESYSTEM GETSPECIALDIR method, due to the fact that it is very Windows-specific – the GET_SPECIAL_FOLDER stored procedure should be used instead.)

The Utility stored procedure (Deprecated)

As long-time readers of this blog will know, this stored procedure was deprecated some years ago at the outset of the OpenInsight 10 project, and it is now basically a thin wrapper around several SYSTEM and FILESYSTEM object methods. It supports the following directory management methods:

  • MAKEDIR
  • REMOVEDIR
  • RENAMEDIR

When called in Event Context each of these methods forwards the request to the FILESYSTEM object. When called outside of Event Context these methods call the following stored procedures instead:

  • UtilityMakeDir
  • UtilityRemoveDir
  • UtilityRename

See below for more details on these.

The RTI_OS_Directory stored procedure (Deprecated)

This stored procedure was deprecated in favor of RTI_OS_Dir as it is very platform specific and makes use internally of RTI_OS_Dir, Utility, and an OLE interface to the Windows Shell, making it less performant than calling those procedures directly. Like Utility it is still supported but will not be updated further with any new functionality.

DirExists stored procedure (Deprecated)

This stored procedure is a simple wrapper around the RTI_OS_Dir DIREXISTS method and so has been deprecated, but can still be used by existing code.

MkDir stored procedure (Deprecated)

This venerable stored procedure is a DLL function that calls the Windows API CreateDirectory function. It has been deprecated in favor of the RTI_OS_Dir MAKEDIR method but can still be used by existing code.

UtilityMakeDir, UtilityRemoveDir and UtilityRename stored procedures (Deprecated)

These three functions form part of the original internals of the Utility stored procedure, and use a “raw” C interface that requires explicit null-terminated strings. They are all still available to use, but are deprecated in favor of the RTI_OS_Dir MAKEDIR, REMOVEDIR and RENAMEDIR methods.

Conclusion

As you can see, version 10 has pulled together the many different historical methods of directory management and consolidated them into two entities that can be used dependent on the execution context. These should be the preferred methods of directory management in your OpenInsight applications.

(As of the time of writing the full documentation for RTI_OS_DIR and GET_SPECIAL_FOLDER is in preparation and will be available shortly. All methods described here apply to version 10.2.3 and later.)

(EDIT: Full documentation for RTI_OS_DIR and GET_SPECIAL_FOLDER is now live on the Revelation Wiki.)

Quickdex in OpenInsight 10

Overview

QUICKDEX.MFS and RIGHTDEX.MFS are programs that provide a quick method to obtain a list of all keys in a table, and both call the subroutine LRDEX.SUB. This stored procedure maintains a sorted list of keys in a special record named %RECORDS%. You can read this list in one operation. For example:

Keys = Xlate(‘MYTABLE’,’%RECORDS%’, ”, ‘X’)

However, there is a cost to maintaining this record. LRDEX.SUB must lock, read, scan, and if needed update the %RECORDS% row every time a row is written or deleted. This creates a potential bottleneck in a busy table. READNEXT operations must scan the result list to remove ‘%RECORDS%’.  The read method must rebuild the %RECORDS% row if it does not exist. The special %RECORDS% row is not compatible with other BFS’s. The LRDEX.SUB program originated with early versions of Advanced Revelation. At that time, storing a list of keys offered a substantial performance benefit. The benefit justified the cost for smallish tables that rarely change. These tables often populate dropdowns for data entry forms, where a quick read of all keys is useful.

When reviewing LRDEX.SUB for OpenInsight 10 we realized it needed revision. The sorting depended on the Arev language sets feature and was incompatible with DICT.MFS, causing crashes.  Rebuilding the %RECORDS% rows was not optimized. We wanted to address these and other issues and improve performance.

Changes

We started by optimizing the %RECORDS% rebuild. We needed to perform a rapid table scan and write a sorted list of keys.  OI10’s RList already provides optimized table scans, so we reused that with V119 to sort the list.  Surprisingly, reading %RECORDS% with and without the new rebuild showed a negligible performance difference!  We realized that storing a list of keys no longer offered much benefit.  We could remove the overhead of maintaining the list without sacrificing performance.  Thus, the revised LRDEX.SUB will perform a table scan when you ask for %RECORDS%. It does not lock or write anything. It still checks for %RECORDS% in a READNEXT. It will delete the record if it finds it. Quickdex.mfs remains best suited to smaller tables. However, it will not fail unexpectedly as a table grows, and it will work with any of the BFS’s.

Sharing data with OpenInsight 9

Note that the OI10 version of LRDEX.SUB is not compatible with the OI 9 version. OI9 requires the %RECORDS% row, OI10 deletes it. For that reason we soft-coded LRDEX.SUB so you can specify which version to use with a configuration setting.  If you need share tables between OI9 and OI10 you must configure OI10 to use the same logic as OI9.  Create a record in the SYSENV table named “RTI_CFG_LRDEX” with “RTI_LRDEX_SUB_9” in the first row.

Summary

OpenInsight 10 offers a simplified Quickdex/Rightdex capability with similar benefits and without the shortcomings of the OI9 version. The simplified version cannot be used on tables shared with OI9. You can configure OI10 to use the OI9 version if needed.

String comparison in OpenInsight – Part 4 – String-Only operators

Basic+ is a dynamically-typed language, and sometimes the system will attempt to change the type of a variable before using it in an operation. For example, if we have variable containing a numeric value as a string, and we attempt to compare it to a variable containing an actual numeric value, then the former will be coerced to a numeric format before the comparison, e.g.:

   StrVar = "3"  ; // String representing an integer
   NumVar = 7    ; // Integer

   Ans = ( StrVar = NumVar ) ; // Ans is FALSE$ and StrVar 
                             ; // is converted to an Integer

This works well in the vast majority of cases, but causes issues when attempting to compare strings that that could be numbers but really aren’t for the purposes of the comparison. For example, consider the following:

   Var1 = "12"    ; // String
   Var2 = "012"   ; // String

   Ans = ( VarA = VarB ) ; // Ans = TRUE$ - should be FALSE$!!

The system first attempts to coerce the operands to a numeric format, and of course it can because they both evaluate to the number 12, and are therefore treated as equal. If you wish to compare them both as strings, the usual solution has been to change them so that they cannot be converted to a number, usually by prefixing them with a letter or symbol like so:

   Var1 = "12"    ; // String
   Var2 = "012"   ; // String

   Ans = ( ( "X" : VarA ) = ( "X" : VarB ) ) ; // Ans = FALSE$

Although this works it hurts performance and can be a cumbersome solution.

OpenInsight 10.2 introduces a new set of extended comparison operators to Basic+ that ensure both operands are coerced to a string type (if needed) before being evaluated, and provides the following benefits:

  • Removes the need for any concatenation thereby avoiding unnecessary copying
  • Removes the need to check if a variable can be a number before the comparison
  • Makes it obvious that you are performing a “string-only” comparison so your code looks cleaner.

The new (case-sensitive) operators are:

OperatorDescription
_eqsEquals operator
_nesNot Equals operator
_ltsLess Than operator
_lesLess Than Or Equals operator
_gtsGreater Than operator
_gesGreater Than Or Equals operator

There is also matching set of case-insensitive operators too:

OperatorDescription
_eqscCase-Insensitive Equals operator
_nescCase-Insensitive Not Equals operator
_ltscCase-Insensitive Less Than operator
_lescCase-Insensitive Less Than Or Equals operator
_gtscCase-Insensitive Greater Than operator
_gescCase-Insensitive Greater Than Or Equals operator

So we could write the previous example like so:

   Var1 = "12"    ; // String
   Var2 = "012"   ; // String

   Ans = ( VarA _eqs VarB ) ; // Ans = FALSE$

Hopefully you’ll find this a useful addition when performing string-only comparisons in your own applications.

The DIRWATCHER control

With the release of version 10.1 a new control type called DIRWATCHER (“Directory Watcher”) has been added to OpenInsight. This is a fairly simple control which allows you to monitor one or more directories on your system and then receive notifications when the contents are changed.

Using the control is very straightforward:

  • Use the WATCHDIR method to add a directory to monitor for changes.
  • Handle the CHANGED event to receive notifications of directory changes.
  • Use the STOP method to stop monitoring directories.

We’ll take a quick look at each of these methods and events below along with a couple of important properties:

The WATCHDIR method

This method allows you to specify a directory to monitor along with some optional flags. It may be called multiple times to watch more than one directory.

bSuccess = Exec_Method( CtrlEntID, "WATCHDIR", DirName, bSubtree, Flags )
ParameterDescription
DirName(required) Specifies the name of the directory to watch. It should be a fully qualified, non-relative directory path.
bSubtree(optional) Set to TRUE$ to monitor all sub-directories beneath DirName as well. Defaults to FALSE$.
Flags(optional) A bit-mask value containing a set of flags that denote the events to monitor in the directories. It defaults to the following flags:

FILE_NOTIFY_CHANGE_FILE_NAME$
FILE_NOTIFY_CHANGE_LAST_WRITE$
FILE_NOTIFY_CHANGE_CREATION$

The flag values are specified in the MSWIN_FILENOTIFY_EQUATES insert record,

This method returns TRUE$ if successful, or FALSE$ otherwise.

The STOP method

This method stops the control monitoring its specified directories.

bSuccess = Exec_Method( CtrlEntID, "STOP" )

This method returns TRUE$ if successful, or FALSE$ otherwise.

(Note – to resume directory monitoring after the STOP method has been called the WATCHDIR method(s) must be executed again).

The CHANGED event

This event is raised when changes have been detected in the monitored directories.

bForward = CHANGED( NewData )

This event passes a single parameter called NewData which contains an @vm-delimited list of changed items (i.e. notifications). Each item in the list comprises an “action code” and the name and path of the affected file, delimited by an @svm.

Action codes are defined in the MSWIN_FILENOTIFY_EQUATES insert record like so:

   equ FILE_ACTION_ADDED$               to 0x00000001   
   equ FILE_ACTION_REMOVED$             to 0x00000002   
   equ FILE_ACTION_MODIFIED$            to 0x00000003   
   equ FILE_ACTION_RENAMED_OLD_NAME$    to 0x00000004   
   equ FILE_ACTION_RENAMED_NEW_NAME$    to 0x00000005 

Remarks

If a monitored directory experiences a high volume of changes (such as copying or removing thousands of files) it could generate a correspondingly high number of CHANGED events, which in turn could produce an adverse affect on your application and slow it down. In order to deal with this potential issue it is possible to “bundle up” multiple notifications with the NOTIFYTHRESHOLD property into a single CHANGED event so they may be processed more efficiently.

The NOTIFYTHRESHOLD property

The NOTIFYTHRESHOLD property is an integer that specifies the maximum number of notifications that should be bundled before a CHANGED event is raised.

CurrVal = Get_Property( CtrlEntID, "NOTIFYTHRESHOLD" )
PrevVal = Set_Property( CtrlEntID, "NOTIFYTHRESHOLD", NewVal )

By default it is set to 100.

The NOTIFYTIMER property

The NOTIFYTIMER property is an integer that specifies the number of milliseconds before a CHANGED event is raised if the NOTIFYTHRESHOLD property value is not met.

CurrVal = Get_Property( CtrlEntID, "NOTIFYTIMER" )
PrevVal = Set_Property( CtrlEntID, "NOTIFYTIMER", NewVal )

By default it is set to 50 (ms).

Remarks

If the NOTIFYTHRESHOLD property is set to a value greater than 1 then the control will try to bundle that number of notifications together before raising a CHANGED event. However, when this is set to a high value it is possible that the threshold may not be reached in a timely fashion and the CHANGED event not actually raised.

E.g. If the NOTIFYTHRESHOLD is set to 1000, and only 200 notifications are received then the CHANGED event would not be raised.

To prevent this problem the NOTIFYTIMER property may be used to specify the amount of time after receiving the last notification before a CHANGED event is raised even if the NOTIFYTHRESHOLD is not met.

E.g. in the example above, if the control had a NOTIFYTIMER of 50, then a CHANGED event would be raised 50ms after the last notification (200) was received, even though the NOTIFYTHRESHOLD of 1000 has not actually been met.

Developer Notes

The DIRWATCHER control is intended as a “non-visual” control and should probably be hidden at runtime in your own applications. However, it is actually derived from a normal STATIC control so all of the properties and methods that apply to a STATIC apply to the DIRWATCHER as well, and you may use them as normal if you wish.

Yield or Die!

When executing a long running process in a desktop environment, such as selecting records from a large table, it is important for the user to be able to interact with the application even though it is busy. A common example of this is displaying a dialog box that shows the progress of an operation and allowing the user to cancel it if they wish. Failure to provide this ability generally results in user frustration, or causes the dreaded “Window Ghosting” effect where Windows changes a form’s caption to “Not Responding” (this is never a good look, and usually ends in a quick visit to the Task Manager to kill the application).

In order to avoid this problem we have to allow the application to check for user interaction, a process usually referred to as “yielding” (hence the awful title of this post), and this time we’ll take a look at the various options available to accomplish this and the differences between them. Before we go any further however, here’s a little background information on how OpenInsight runs beneath the hood so that you can appreciate how messages and events are handled.

Under the hood

OpenInsight.exe (aka. the “Presentation Server”, or “PS”) has a main thread (the “UI thread”) with a Windows message loop that manages all of the forms and controls, and it also has an internal “event queue” for storing Basic+ events that need to be executed. The PS also creates an instance of the RevEngine virtual machine (“the engine”), which has its own thread (the “engine thread”) with a Windows message loop, and is responsible for executing Basic+ code.

When the PS needs to execute an event it passes it to the engine directly if possible, otherwise it adds the details to the event queue and then posts a message to itself so the queue can be checked and processed when the engine is not busy. When the engine receives the event data it is executed on the engine thread. Stored procedures such as Get_Property, Set_Property, and Exec_Method provide a way for the Basic+ event to communicate back to the PS to interact with the user interface controls and forms during its execution.

The key point to note here is that Basic+ event code runs in a different thread to the UI, so while the engine thread is processing the event, the UI thread is basically waiting for it to finish, and this means that it may or may not get chance to process it’s own message loop. This is where the problems can begin, and why the need for a yielding ability, because:

  1. The engine thread needs to be paused or interrupted in some fashion so that the UI thread can check and process its own Windows message queue for things like mouse, keyboard and paint messages. If this queue is not checked at least every 10 seconds then Windows assumes that the PS is hung and the “Not Responding” captions are shown on the application forms.
  2. While the engine is processing an event, the PS cannot pass it a new one, so it is added to the event queue. If we are waiting to process some Basic+ event code like a button CLICK to cancel the current operation, then we need some way for this to be retrieved and executed before the current event is finished.

So, now we know why “window ghosting” happens we can take a look at the various options to deal with it.

Options for yielding

MSWin_Sleep stored procedure

This is a direct call to the Windows API Sleep function, and it puts the engine thread to sleep for at least the specified number of milliseconds. However, while calling this will allow Windows to schedule another thread to run, there’s no guarantee that this would be the UI thread, so it’s not really a good solution.

WinYield stored procedure

This is a simple wrapper around the Windows API Sleep function, with a sleep-time of 10ms. This suffers from the same disadvantages discussed for MSWin_Sleep above (This function remains for backwards compatibility with early versions of of OI and Windows).

MSWin_SwitchToThread stored procedure

This is a direct call to the Windows API SwitchToThread function which forces Windows to schedule another thread for execution. Like MSWin_Sleep and WinYield there’s no guarantee that this would cause the UI thread to run, so again it’s not a great solution.

SYSTEM PROCESSEVENTS method (a.k.a Yield stored procedure)

This is a new method in version 10.1 that performs two tasks that solve the problem:

  1. It explicitly tells the UI thread to process its message queue (which will avoid the “ghosting” issue), and
  2. It allows the UI thread to process the event queue so waiting events can be executed as well.

One possible drawback here is that waiting events are also processed, and this might not be a desirable outcome depending on what you are doing. In this case there is another method called PROCESSWINMGS that should be used instead.

(FYI – The PROCESSEVENTS method is essentially a wrapper around the venerable Yield() stored procedure, but allows the same functionality to be called via the standard object-based method API rather than as a “flat” function. Yield() itself is still and will be fully supported in the product).

SYSTEM PROCESSWINMSGS method

This is a new method in version 10.1 that tells the PS to process it’s Windows message queue but it does not process any Basic+ events, i.e. it prevents the “ghosting” effect but does not cause events to fire before your application is ready for them.

Conclusion

Version 10.1 has added more functionality to help you avoid the dreaded “Not Responding” message via the PROCESSWINMGS and PROCESSEVENTS methods, and hopefully, armed with the information above, this will help you to write better integrated desktop-applications.