Monthly Archives: July 2021

The Saga of ShellExecute

One of the most popular “raw” Windows API functions that OpenInsight developers have used over the years is the ShellExecute function, which allows you to launch an application via its filename association, e.g. you can launch Word by using a document file name, or Excel using a spreadsheet filename and so on.

However, because it was never really made an “official” part of the product (it was normally passed on in forums), developers were left to create their own DLL Prototype definitions in order to use it – this gave rise to many variations over the years, many of which were not compatible with others. For example, some use LPCHAR as an argument type, some use LPSTR or LPASTR, whilst others use LPVOID with GetPointer(); some definitions use the “Wide” version of the function, some the “Ansi” version, and there are many different aliases, with or without the “A/W” suffix too. The list goes on.

For OpenInsight 10 we decided that we couldn’t move forward with this as we would run the risk of conflicting with established applications, so we moved all of the DLL Prototypes we used into a new namespace called “MSWIN_” and claimed it as our own. This left developers to bring forward their own DLL prototypes into version 10 as and when needed, and therefore we didn’t supply a “ShellExecute” function as such, though we did supply “MsWin_ShellExecute” instead (see below).

Another decision we took was to try and move away from the need for developers to use raw Windows API function calls as much as possible, as some of them can be complex and require knowledge of C/C++ programming, which is not necessarily a skill set that everyone has the time or desire to learn. Ergo, we moved a lot of functionality into the Presentation Server (PS) and created some Basic+ wrapper functions around others to shield developers from the sometimes gory internals.

(We also chose to use the “W” versions of functions rather than the “A” versions where possible, because these would translate better when in UTF8 mode and remove the need for an extra “A”->”W” conversion in Windows itself.)

So, coming back to ShellExecute, and in light of the above, we have three “official” and supported ways of calling it in OpenInsight 10 as detailed below:

  • The SYSTEM object SHELLEXEC method
  • The RTI_ShellExecuteEx stored procedure
  • The MSWin_ShellExecute DLL Prototype stored procedure

The SYSTEM object SHELLEXEC method

If your program is running in “Event Context”, (i.e. it is executing in response to an event originating from the PS) then you may use the SYSTEM SHELLEXEC method which invokes ShellExecuteW internally.

RetVal = Exec_Method( "SYSTEM", "SHELLEXEC", OwnerForm, Operation, File, |
                      Parameters, WorkingDir, ShowCmd )
OwnerFormNoName of a form to use as a parent for displaying UI messages.
OperationNoOperation to be performed; “open”, “edit”, “print” etc.
FileYesFile to perform the operation on.
ParametersNoIf File is an executable file this argument should specify the command line parameters to pass to it.
WorkingDirNoThe default working directory for the operation. If null the current working directory is used.
ShowCmdNoDetermines how an application is displayed when it is opened (as per the normal VISIBLE property).

The return value is the value returned by ShellExecuteW.

The RTI_ShellExecuteEx method

This stored procedure is a wrapper around the Windows API ShellExecuteExW function (which is used internally by ShellExecuteW itself), and may be used outside of event context – it can also return the handle to any new process it starts as a result of executing the document. As you can see it’s quite similar to the SHELLEXEC method:

RetVal = RTI_ShellExecuteEx( Hwnd, Verb, File, Parameters, |
                             Directory, nShow, hProcess )
HwndYesHandle of a window to use as a parent for displaying UI messages, or null (0) to use the desktop.
VerbNoOperation to be performed; “open”, “edit”, “print” etc.
FileYesFile to perform the operation on.
ParametersNoIf File is an executable file this argument should specify the command line parameters to pass to it.
DirectoryNoThe default working directory for the operation. If null the current working directory is used.
nShowNoDetermines how an application is displayed when it is opened (as per the normal VISIBLE property).
hProcessNoReturns the handle to the new process.

The return value is the value returned by ShellExecuteExW.

The MSWin_ShellExecute DLL Prototype stored procedure

This is the “raw” DLL function that is included with OI10, and the definition can be found in the MSWIN_SHELL32 DLLPROTOTYPE entity:


Because we’re using LPWSTR data types there is no need to null-terminate any of your variables so using it is quite simple:

RetVal = MsWin_ShellExecute( 0, "open", "stuff.docx", "", "c:\docs", 1 )

Migrating ShellExecute

Whilst you are free to use one of the methods outlined above, this may not be optimal if you are still sharing code between your existing version 9 application and your new version 10 one. In this case there are a couple of options you could use:

  • Define your preferred DLL prototype in v10.
  • Use a wrapper procedure and conditional compilation.

Defining your own prototype

This is probably the easiest option – you simply use the same prototype in v10 that you did in version 9, with the same alias (if any), and this way the code that uses it doesn’t need to be changed. The only downside to this if you’ve used any 32-bit specific data types instead of 32/64-bit safe types like HANDLE (this could happen if you have a really old prototype) – you must ensure that you use types that are 64-bit compliant.

Using conditional compilation

This is a technique we used when writing the initial parts of v10 in a v9 system so our stored procedures would run the correct code depending on the platform they were executing on (it was actually first used to share code between ARev and OI many years ago!).

The v10 Basic+ compiler defines a token called “REVENG64” which is not present in the v9 compiler – this means that you can check for this in your source code with “#ifdef/#ifndef” directives and write code for the different compiler versions.

For example, you could write your own wrapper procedure for ShellExecute that looks something like this:

Compile Function My_ShellExecute( hwnd, verb, file, params, dir, nShow )

#ifdef REVENG64
   // V10 Compiler - use RTI function
   Declare Function RTI_ShellExecuteEx
   RetVal = RTI_ShellExecuteEx( hwnd, verb, file, params, dir, nShow, "" )

#ifndef REVENG64
   // V9 Compiler - use existing "raw" prototype
   Declare Function ShellExecute
   RetVal = ShellExecute( hwnd, verb, file, params, dir, nShow )

Return RetVal

And then call My_ShellExecute from your own code.

So, there ends the Saga of ShellExecute … at least for now.

Methods, Events, and Documentation

In a recent post we provided a preview of the OpenInsight IMAGE API documentation for the upcoming release of version 10.1. As that proved quite popular we thought we’d provide some more, this time dealing with the Common GUI API (i.e. the basic interface that virtually every GUI object supports) and the WINDOW object API – two core areas of OI GUI programming.

Methods, not Events

One thing you may notice as you look through these documents is the addition of many new methods, such as SHOWOPTIONS or QBFCLOSESESSION – this is an attempt to tidy up the API into a more logical and coherent format that is a better fit for an object-based interface.

As we went through the product in order to document it, it became very apparent that there were many instances where events were being used to mimic methods, such as sending a WRITE event to save the data in a form, or sending a CLICK event to simulate a button click and so on. In object-based terminology this sort of operation would be performed by a method, which is a directive that performs an action – the event is a notification in response to that action. So, for example, you would call a “write” method to save your data and the system would raise a “write” event so you could deal with it.

Of course, this distinction will probably not bother many developers – just API purists like myself, but this does have another advantage if you like to use Object Notation Syntax (I do) – you can now perform actions such as reading and writing form data by using the”->” notation, whereas before you would have to use the Send_Event stored procedure which essentially breaks the object-based paradigm.

So instead of:

   Call Send_Event( @Window, "WRITE" )

you would use the form’s WRITEROW method instead:

   @@Window->WriteRow( "" )

which is a more natural fit for this style of programming.

(It is also easier to explain to new OI programmers who are used to other object-based languages and environments where everything is properties, methods and events).

Methods, not Stored Procedures

This brings us finally onto the topic of Stored Procedures and the object API, where several of these also fulfill the role of methods. For example, take the venerable Msg stored procedure used to display a message box for a parent form – a different way of treating this would be to have a SHOWMESSAGE method for the parent form rather than using a “raw” Msg call. Likewise for starting a new form: instead of using the raw Start_Window procedure, the SYSTEM and WINDOW objects now support a STARTFORM method instead.

Of course, none of this changes your existing code, nor is it enforced, it’s just something you can use if and when you wish to. However, even if my API pedantry hasn’t persuaded you to change your coding style, some of the new methods are worth investigating as they provide a better opportunity for us to extend the product’s functionality further – take a look at the WINDOW READROW and WRITEROW methods for an example of this – they support new features that we couldn’t do with just sending events.

In any case, here are the links – hopefully some light reading for your weekend!