Author Archives: Captain C

Profile logging updates in OpenInsight 10.2

Previous releases of OpenInsight have included a simple logging system that records all stored procedure calls and the time taken to execute them. This logging system is enabled by creating a blank file called “RevProfile.log” in the OpenInsight directory before the application is started. If this file is present when the system boots it creates a new log file (with a name based on the application’s Windows Process ID) and writes all stored procedure activity to it until the application is stopped.

Whilst this can be a useful tool it does suffer from a few issues:

The log file records all activity. This usually results in very large files that are difficult to navigate and process, making them very tedious to use.
There is no facility to annotate parts of the log, so it can be difficult to narrow down the actual section of code that needs to be profiled.
The profile timing uses the classic Windows “tick count” which has a granularity of 16ms. Many stored procedures take much less time to execute than this, so entries in the log can be wildly inaccurate, usually appearing as “0ms” or “16ms”.
There are no tools to actually parse the log so that it can be analyzed more easily.

With this release of OpenInsight we have provided some new functionality to address these issues, namely:

The “RevProfileLog” function, and
The “RevProfile Log Analyzer” form.

In this post we’ll take a look at these additions and examine how they can help in your application development.

The RevProfileLog function

This is a new stored procedure for version 10.2 that allows much greater control of the profile logging process and offers the following benefits:

It can be started and stopped as needed, allowing the scope of recording to be narrowed to a critical area (we call this “Dynamic Logging“, as opposed to the old way of profile logging mentioned above, which we call “Static Logging“).
The log file can be created using a custom name and location.
The file can be appended to each time it is started, or cleared.
The log can exclude specific stored procedures.
The log can record notes against a specific stored procedure to help identify a particular call in the log.
The log can record arbitrary text in the file to help with subsequent analysis.

All of this is controlled via the new RevProfileLog stored procedure (this is compiled as a subroutine so has no return value):

Call RevProfileLog( Method, Param1, Param2 )

The Method parameter is a numeric value that specifies the logging operation to perform; the Param1 and Param2 parameters depend on Method as described below:

Method	Description	Param1	Param2
1	Start Log: Starts the profile logging activity.	The name of the log file to use or create.	A boolean flag – if TRUE$ then the log file is cleared before use. Defaults to FALSE$.
2	Stop Log: Stops the profile logging activity.	N/a.	N/a.
3	Set Note: Appends text to the next stored procedure call written to the log. It is reset to null afterwards.	The note text.	N/a.
4	Output Text: Outputs text to the log.	The text to output.	The indent to output to. Specify “-1” (the default) to output at the current indent, otherwise specify a value between 1 and 256 if desired.
5	Exclude Proc: Prevents a stored procedure from being included in the log file.	The name of the stored procedure (upper- case).	N/a.
6	Include Proc: Removes a stored procedure from the list of excluded stored procedures so that it may be written to the log again.	The name of the stored procedure (upper-case).	N/a.
7	Clear Excluded: Clears the excluded stored procedure list.	N/a.	N/a.

Equates for these methods can be found in the RTI_REVPROFILE_EQUATES insert record.

Example: Profiling a call to Get_Property

Compile Subroutine Test_RevProfileLog( void )

   Declare Function Get_Property
   $Insert RTI_RevProfileLog_Equates
   $Insert Logical
   
   LogFile = "c:\temp\test_revprofile.log"
   
   // Create the log file and clear it before it is written to:
   Call RevProfileLog( RPL_STARTLOG$, logFile, TRUE$ )
   
   // Let's not see any RTP57/A calls (if any)
   Call RevProfileLog( RPL_EXCLUDEPROC$, "RTP57" )
   Call RevProfileLog( RPL_EXCLUDEPROC$, "RTP57A" )
   
   // Add a note so we can see why we are calling Get_Property
   Call RevProfileLog( RPL_SETNOTE$, "Getting SYSTEM VERSION" )
   Version = Get_Property( "SYSTEM", "VERSION" )
   
   // Add some text to the log, but don't indent it.
   Call RevProfileLog( RPL_OUTPUTTEXT$, "Done with this log now ***", 0 )
   
   // Stop the log
   Call RevProfileLog( RPL_STOPLOG$ )

Return

Produces the following log file:

		GET_PROPERTY 0.00 Getting SYSTEM VERSION
			PS_GET_PROPERTY 0.12
				OEREQINFO 0.15
				OEREQINFO 2.29 2.14
			PS_GET_PROPERTY 2.31 2.19
			PS_GET_PROPERTY 2.32
				OEREQINFO 2.32
				OEREQINFO 2.42 0.10
			PS_GET_PROPERTY 2.43 0.12
			PS_GET_PROPERTY 2.44
				OEREQINFO 2.44
				OEREQINFO 2.53 0.09
			PS_GET_PROPERTY 2.55 0.11
			PS_GET_PROPERTY 2.55
				OEREQINFO 2.56
				OEREQINFO 2.65 0.09
			PS_GET_PROPERTY 2.66 0.11
			RTP65 2.67
			RTP65 2.67 0.01
			PS_GET_PROPERTY 2.68
				OEREQINFO 2.69
				OEREQINFO 2.78 0.09
			PS_GET_PROPERTY 2.79 0.11
		GET_PROPERTY 2.80 2.80
>> Done with this log now ***

Things to note:

Text lines added with the “Output Text” method are prefixed with a “>>” string to denote that they are text.
The timings now use the Windows high resolution “QueryPerfomanceCounter” functions to record timings, so the results are far more accurate and we’re now reporting at a 100th of a millsecond.
The indenting does not begin at “level 1” because the system call stack was already several levels in before logging was started.
Calls to RevProfileLog itself are always excluded.
As we mentioned above we call the original method of profile logging “static logging” (i.e. create a file called “RevProfile.log” and record everything). This feature still works in the same way, but with the added bonus of using the improved timings and respecting all of the RevProfileLog function methods except for the “Start Log” and “Stop Log” methods. They are ignored during static logging.

The RevProfile Log Analyzer form

This is a new form (called RTI_IDE_ANALYZE_PROFILELOG) that can parse a RevProfile log file and present the information in a format that is easy to use.

For example, here is the small log file example shown above processed and loaded into the form:

An image of the RevProfile Log Analyzer form (RTI_IDE_ANALYZE_PROFILELOG).

Using this form is fairly simple: just select the log file you want to analyze and wait for the form to parse and process the information. Once this has been done the data from the log is presented in two sections:

The Call Trace Log (on the left), and
The Stored Procedure Call Details (on the right)

The Call Trace Log

This is a TreeListBox control that shows the sequence of calls along with the time taken for that call. It mirrors the structure of the actual log file and displays text notes using a different icon (a green globe), rather than the blue one used for stored procedure calls.

Double-clicking on a stored procedure call selects it in the Stored Procedure Call Details section on the right.

The Stored Procedure Call Details

This section comprises two EditTable controls:

The top one contains a list of all the stored procedures found in the log, along with the number of times each was was executed and the total time spent executing.
The bottom one details the individual calls to the selected stored procedure – each entry in the table shows the index of the call in the Call Trace Log, the line number in the actual log file where the call was recorded, and the time taken by that call. Double-clicking a call selects the matching entry in the Call Trace Log opposite.

Things to note

Both edit tables support sorting via double-clicking on the appropriate column header.
Annotated stored procedure calls (i.e those with a note added to them via the RevProfileLog “Set Note” method) are separated from calls to the same stored procedure that are not annotated. This is to ensure that it is easy to isolate and find them in the output.

Very large log files – a cautionary note

RevProfile log files can grow rather large, especially the “static” log files that record the entire lifetime of the application. Whilst these files can be easily read and parsed in smaller chunks, representing their call trace structure in a something like a TreeListBox control can be challenging due to the time taken to load items. For example, something like a 15MB log file with nearly 200,000 calls may take several seconds to appear in the control once the log has been parsed and we set the control’s LIST property (i.e. “Loading call trace list …” appears in the progress bar), and the UI may appear frozen. This is not the case – if you are loading a very large file be sure to exercise a little patience before you reach for the Task Manager!

Conclusion

The new RevProfileLog function provides much better and much-needed profile logging functionality to OpenInsight, making the information much easier to use, and we’re sure it will help you improve the performance of your applications.

(This functionality is available in OpenInsight 10.2 from the Beta 3 release onwards)

EditTables – The CELLPOSCHANGED event

Leave a reply

OpenInsight 10.2 adds a new event called CELLPOSCHANGED to the EDITTABLE control. This is effectively the same as the normal POSCHANGED event but with the addition of an extra parameter called “ContextFlags” that provides more information on why the event was raised.

bForward = CELLPOSCHANGED( CtrlEntID,   |
                           CtrlClassID, |
                           NextColumn,  |
                           NextRow,     |
                           ContextFlags )

ContextFlags is a simple bitmask integer that contains the following flags:

Bit Flag Value	Description
0x00000001	If set then the cell position was changed via a keystroke.
0x00000002	If set then the cell position was changed via the mouse.

Equates for these flags can be found in the PS_EDITTABLE_EQUATES insert record:

Equ PS_EDT_CTF_NONE$         To 0x00000000;
Equ PS_EDT_CTF_KEYSTROKE$    To 0x00000001;
Equ PS_EDT_CTF_MOUSECLICK$   To 0x00000002;

Example – testing to see if the position (CARETPOS) was changed via a mouse click:

$Insert PS_EditTable_Equates

If BitAnd( ContextFlags, PS_EDT_CTF_MOUSECLICK$ ) Then
   // CARETPOS was changed by using the mouse.
   ...
End

Notes on using the CELLPOSCHANGED event

The default promoted system CELLPOSCHANGED event handler performs the same processing as the default promoted system POSCHANGED event handler (i.e. data validation and required checking etc).
If a CELLPOSCHANGED event handler is defined by the developer then a standard POSCHANGED event will not be raised.
To preserve backwards compatibility with existing applications the default promoted system CELLPOSCHANGED event will not be compiled into a control if there is no CELLPOSCHANGED quick event handler. This is to ensure that POSCHANGED is always executed if CELLPOSCHANGED has not been explicitly set for a control by the developer.
CELLPOSCHANGED is available in OpenInsight 10.2 from the Beta 3 release onwards.

Migration tool updates in OpenInsight 10.2

Leave a reply

Based on user feedback (and our own experience) version 10.2 includes the following new updates for the OpenInsight migration process that we think you’ll find invaluable when moving your applications:

The ability to choose individual migration tasks
The ability to select individual entities
The ability to migrate SYSPROG components

We’ll take a look at each of these features below, along with a small note on “migrating” your data.

Task Selection

In previous releases the Migration Tool had a fixed set of tasks to perform which it did in sequence. In this release you may now specify which tasks you wish to execute instead, which might save some time if you have to repeat the migration process more than once (a common requirement if needing to keep a version 9 application synchronized with version 10 during upgrade development).

By default all tasks are selected, but you may deselect any that you don’t wish to run.

Task	Description
Migrate App Settings	Migrates information from the SYSAPPS record. Migrates database environment settings. Migrates custom event (CFG_EVENTS) settings.
Migrate Users	Migrates user details into the SYSUSERS table.
Migrate DLL Prototypes	Migrates “DLL_” records in SYSPROCS to DLLPROTOTYPE entities.
Migrate C-Structures	Migrates “STRUCT_” records in SYSOBJ to DLLSTRUCT entities.
Migrate Promoted Events	Migrates promoted event records in SYSREPOSEVENTS/EXES to PROMOTEDEVENT/EXE entities.
Migrate RDK records	Migrates SYSREPOSVIEWS/RELEASES records to REPVIEW/RELEASE entities.
Migrate Entities	Migrates application entities. Note that these can now be specified individually if desired (see below).
Migrate Arev32 records	Migrates Arev32-specific records in SYSENV. Migrates Arev32 records from SYSPRINTERS.
Clean Relationships	Removes invalid relationships (Using/Used By).
Compile Stored Procedures	Compiles migrated stored procedures.
Compile Promoted Events	Compiles migrated promoted events.
Compile Forms	Compiles migrated forms.

Migration Tool Tasks

Entity Selection

If you choose the “Migrate Entities” task then you now have the option to select individual entities to migrate (If you don’t select any then all entities in the v9 application will be migrated).

To select individual entities right-click on the “Selected Entities” control and choose “Select Entities” from the menu, or click the “…” options button in the top-right of the control’s header:

You may now use the “Select Repository Entities” dialog to choose which entities you wish to migrate:

As you can see there are quite a few options you can use to find the entities you wish to migrate – these options make a “repository filter” that is used to create a list to choose from:

Option	Description
Filter ID	Name of a saved filter (REPFILTER entity). Choosing a filter will load the other selection criteria controls in this dialog with the saved filter details.
View ID	Name of a saved Repository View (REPVIEW entity). If specified, the list of entities in this View is used as a starting point for the filter when it executes.
Entity ID	If specified only entities matching this name will be selected. The standard ‘[]’,’]’ and ‘[‘ RList-style syntax for “containing”, “starting with” and “ending with” is respected.
Date From	If specified only entities updated from this date will be included. This option supports a “nD” syntax where “n” is the number of days since the current date. E.g. to select entities updated since the previous day specify “1D”, for entities updated in the last week specify “7D” and so on.
Date To	If specified do not include entities after this date. This option supports the same “nD” syntax as the “Date From” option.
Updated By	If specified then only entities updated by the specified user(s) are returned. This option may contain a “,” delimited list for matching against multiple users.
Filter Options	Only entities matching the checked options will be returned. Note that for a migration operation the “Show Inherited Entities” checkbox is always disabled (This dialog is used elsewhere in the system to select repository entities, so it can be enabled in such scenarios).
Filter By Types	Allows you to choose specific entity types and classes to include. If blank then all types and classes are included in the search. Click the “…” options button in the top right corner of the control, or right-click to use the context menu to launch a dialog to select the types and classes.
Filter By SELECT	This option allows you to enter an RLIST SELECT statement against the SYSREPOS table that will be used to filter the entities.
Save Filter	Click this button to save the filter criteria for future re-use (stored as a REPFILTER entity in the repository).
Apply Filter	Click this button to execute the filter criteria and load a list of matching entities into the “Available” controls below. If you have not entered any criteria then all entities in the v9 application will be selected.

Filter Options

Once you have applied your filter the “Available Types” control is populated – selecting a Type populates the “Available Entities” control where you can select the items that you wish to migrate. Use the “>” and “>>” buttons to add them to the “Selected Entities” control on the right, or the “<” and “<<” buttons to remove them.

Things to note:

As items in the “Available Entities” control are selected, the list of items that they use is shown in the “Related (Used By) Entities” control below. These can also be selected if you wish.
Each of the “Available” and “Selected” controls have context menus and “…” options buttons (top-right corner).
If the “Auto-Add Executable” option is checked when you add entities to the “Selected Entities” control then any executable versions of the entity (e.g. STPROCEXE, OIWINEXE etc) are also added.
If the “Auto-Add Source” option is checked when you add entities to the “Selected Entities” control then any source versions of an executable entity (e.g. STPROC, OIWIN etc) are also added.

Clicking “OK” takes you back to the main migration form where you will see the list of entities selected. From there you may click the “Next” button to start the migration process.

(Note that the task and entity selections are saved automatically between migrations so that you may use them again if you wish.)

Migrating items from SYSPROG

Previous versions of the migration tool prevented migrating any entities from the SYSPROG application. This was to preserve the system from being damaged in case older v9 entities overwrote the v10 ones, thereby rendering it unusable.

With this release you may run the migration form from your SYSPROG application with the following restrictions:

You may only execute the following tasks:
- Migrate RDK Records
- Migrate Entities – You must specify the entities to migrate – you cannot migrate “All Entities”!
- Migrate Arev32 Records
- Clean Relationships
- Compile Stored Procedures
- Compile Forms

Bonus Trivia – a quick note on migrating data

The migration process begins with creating a new application in version 10 and then executing a form called RTI_MIGRATE_V9_TO_V10, which guides you through the steps necessary to move your application from version 9.

At this point you select the location of the application you are migrating from, after which you see the data volumes that are attached in it. There are options for the data to be “migrated”, but in my experience it is best to move and attach the data you want for your application before you attempt the migration rather than use the facility here. The migration process does not change your data at all so there is no real advantage to using this facility (it is basically provided as a convenience), and in fact, if you are moving tables containing relational indexes they will be removed. It is far easier to copy/move and attach the tables yourself using the Database ToolPanel before you get here.

Conclusion

The migration tool has been significantly improved for the version 10.2 release and we’re sure you will find it much more helpful when moving your older applications to the latest version of OpenInsight.

ListBoxes and TreeListBoxes – “in-place” editing

Leave a reply

One of the new features that was added to the ListBox and TreeListBox controls in version 10 was the ability to use “in-place” editing on the items in the same manner as the Windows Explorer when you press “F2” or double-click an item. If you’ve done any work with the OpenInsight Menu Designers you will have seen this capability in action.

The READONLY property

Enabling in-place editing is as simple as setting the READONLY property to False – once you’ve done that the user can press “F2” while using the control and edit the text of the currently selected item. Pressing “Enter” when editing will update the item text (as will losing focus or selecting another item), while pressing “Esc” will abandon the changes. Obviously that’s a very simple take on the topic and hardly worth a blog post in and of itself, so next we’ll take a look at some of the properties, methods and events that you can use to tailor the editing process.

The EDITING property

This is a simple boolean property that returns TRUE$ if an item is being edited.

The EDITORHANDLE property

This property returns the HANDLE of the editor control if an item is being edited.

The EDITKEY property

By default, hitting “F2” on an item puts the control into “edit mode”, just like the Windows Explorer. However, if you wish to change this then you may use the EDITKEY property to select another key instead. The edit key is a Windows virtual key code and constants for these codes can be found in the MSWIN_VIRTUALKEY_EQUATES insert record.

The EDITOPTIONS property

This property allows you to fine-tune some of the validation options for the editor:

TextCase – Specifies if the text entered is lower-case only, upper-case only or mixed (See the EDITLINE TEXTCASE property for more details).
ValidChars – Specifies which characters may be entered into the editor (See the EDITLINE VALIDCHARS property for more details).
MaxLimit – Specifies the maximum number of characters that may be entered into the editor (See the EDITLINE LIMIT property for more details).

At runtime this property is an @fm-delimited array – constants for use with this property can be found in the PS_LISTBOX_EQUATES insert record.

The INCLUDEEDITORTEXT property

By default getting item text from the ListBox (e.g. via the LIST property) will include the text from an item currently being edited, even if that edited text has not yet been saved. Setting this property to FALSE$ ensures that the returned item text ignores the value in the editor instead.

The BEGINEDIT method

This method allows you to programmatically put the ListBox into edit mode, as if the user had pressed “F2” (or whatever value the EDITKEY property is set to). You may specify the index of the item to edit, otherwise it will default to the current item.

RetVal = Exec_Method( CtrlEntID, "BEGINEDIT", itemIndex )

The ENDEDIT method

This method allows you to programmatically stop the item editing process, optionally allowing any changes to be accepted as if the “Enter” key had been pressed (the default is to reject any changes as if the “Esc” key had been pressed).

RetVal = Exec_Method( CtrlEntID, "ENDEDIT", bUpdate )

The ITEMEDITING event

This event is raised when an item it put into edit mode, and passes the index of the item as a parameter:

bForward = ITEMEDITING( CtrlEntID, CtrlClassID, ItemIndex )

The ITEMCHANGED event

This event is raised when the item is updated via the editor, i.e. the user hit the “Enter” key, the control lost the focus, or the EDITEDIT method was used with the bUpdate parameter set to TRUE$. The event passes the index of the item that has changed as well as the old and new data:

bForward = ITEMCHANGED( CtrlEntID, CtrlClassID, ItemIndex, |
                                                NewData,   |
                                                PrevData )

The ITEMUNCHANGED event

This event is raised when an item leaves edit mode without being updated, i.e. the user hit the “Esc” key or the EDITEDIT method was used with the bUpdate parameter set to FALSE$. The event passes the index of the item that was being edited:

bForward = ITEMUNCHANGED( CtrlEntID, CtrlClassID, ItemIndex )

So that wraps up this short post on ListBox editing – we’re sure that you’ll find many useful ways of implementing this new feature when expanding your application’s functionality.

The HTTPSERVER control

2 Replies

OpenInsight 10.2 introduces a new control type called HTTPSERVER, which provides a lightweight HTTP web-server control for use in your applications.

Using the control is a simple process:

Drop a “HTTP Server” control onto a form
Set the port number to listen on via the PORT property
Use the START method to start the server, or set the STARTUPMODE property to “Automatic” if you want the server to start when the form is created.
Listen for requests via the HTTPREQUEST event and respond to them using methods such as the SETRESPONSECONTENT and SETRESPONSEFILE methods
Return the response using the SENDRESPONSE method

So, if you set up a HTTPSERVER on port 8089 and execute the form you can use a browser to communicate with it via a URL something like this:

http://localhost:8089/path1/path2/test?arg1=this&arg2=that

The HTTPREQUEST event

As you can see, setting up the control is fairly easy, but the bulk of the work needs to be done in the HTTPREQUEST event where you examine the contents of the request and return the appropriate content.

The event has the following interface:

   bForward = HTTPREQUEST( CtrlEntID,     |
                           CtrlClassID,   |
                           RequestID,     |
                           RequestHeaders )

And the following parameters:

Name	DESCRIPTION
CtrlEntID	ID of the HTTP Server control firing the event
CtlrClassID	Type of control firing the event – this is always “HTTPSERVER”
RequestID	Unique identifier for returning a response to the client – this is used with the various “SETRESPONSE” methods that set response data.
RequestHeaders	An @FM delimited dynamic array of data that describes the request. The format is similar to the HTTPRequest argument used in OECGI programming. The full format is described in the PS_HTTPSERVER_EQUATES insert record.

As mentioned above, the RequestHeaders parameter describes the details of the request using a format similar to the format used in OECGI programming. There are some differences that are worth highlighting however:

For a GET request the query values are already parsed into their own fields (<37> and <38>) as an associated multi-value pair. They are not found unparsed in field <1> as per OECGI.
For a POST or PUT request the content is obtained using the GETREQUESTCONTENT method (see below) – it is not passed in the RequestHeaders variable.
Cookies are already parsed into their own fields (<39> and <40>) as an associated multi-value pair.
Headers are already parsed into their own fields (<35> and <36>) as an associated multi-value pair.

Note that out of the box we do not enforce any restrictions or framework on how you handle the request – compare this to classic OECGI programming where the “PathInfo” field is used to determine which “INET_” procedure is executed to fulfill it (via the RUN_INET_REQUEST stored procedure) There is no such requirement with the HTTPSERVER control, and you may create your own framework if you wish (although see the note on RTI_RUN_HTTPSERVER_REQUEST below).

Returning a response

There are several methods described below that you may use to process the content that you return to the client.

GETREQUESTCONTENT
GETRESPONSECONTENT
GETRESPONSECOOKIE
GETRESPONSEFILE
GETRESPONSEHEADER
GETRESPONSESTATUS
ISPORTINUSE
SETRESPONSECONTENT
SETRESPONSECOOKIE
SETRESPONSEFILE
SETRESPONSEHEADER
SETRESPONSESTATUS
SENDRESPONSE

Note: With each of these you must use the unique RequestID parameter passed to you in the HTTPREQUEST event.

E.g. Returning HTML in the HTTPREQUEST event

ReposID = @AppID<1> : "*DOC*HTML*INDEX_PAGE" 
HTML = Repository( "ACCESS", ReposID )
Call Exec_Method( CtrlEntID, "SETRESPONSECONTENT", RequestID, HTML )
Call Exec_Method( CtrlEntID, "SENDRESPONSE" )

E.g. Returning a file in the HTTPREQUEST event

ReposID = @AppID<1> : "*DOC*HTML*INDEX_PAGE" 
FilePath = Repository( "GETSUBKEY", ReposID )
Call Exec_Method( CtrlEntID, "SETRESPONSEFILE", RequestID, FilePath )
Call Exec_Method( CtrlEntID, "SENDRESPONSE" )

The HTTPSERVER methods

GETREQUESTCONTENT method

Gets the raw content sent to the server by the client as part of a POST or PUT request.

 ReqContent = Exec_Method( CtrlEntID, "GETREQUESTCONTENT", RequestID )

GETRESPONSECONTENT method

Returns raw content for the response as set via a previous call to the SETRESPONSECONTENT method.

 RspContent= Exec_Method( CtrlEntID, "GETRESPONSECONTENT", RequestID )

GETRESPONSECOOKIE method

Returns details for a response cookie as set via a previous call to the SETRESPONSECOOKIE method (see below for the CookieVal format).

 CookieVal = Exec_Method( CtrlEntID, "GETRESPONSECOOKIE", RequestID, |
                                                          CookieName )

GETRESPONSEFILE method

Returns the name and path of the response content file set with a previous call to the SETRESPONSEFILE method.

 RspFile = Exec_Method( CtrlEntID, "GETRESPONSEFILE", RequestID )

GETRESPONSEHEADER method

Returns the details for a response header as set via a previous call to the SETRESPONSEHEADER method.

 HeaderVal = Exec_Method( CtrlEntID, "GETRESPONSEHEADER", RequestID, |
                                                          HeaderName )

GETRESPONSESTATUS method

Returns the HTTP status code of the response (defaults to 200).

 RspStatus = Exec_Method( CtrlEntID, "GETRESPONSESTATUS", RequestID )

ISPORTINUSE method

Returns TRUE$ if the specified port is in use on the local machine.

 InUse = Exec_Method( CtrlEntID, "ISPORTINUSE", Port, IPv6 )

Port – Identifies the port to check
IPv6 – if TRUE$ then check the IPv6 bindings, otherwise check the IPv4 bindings

SENDRESPONSE method

Sends the response back to the client. This method should be called when you have finished setting the response details (Note that this is called by the promoted system HTTPREQUEST handler in case you forgot to do it in your own code!).

 RspSent = Exec_Method( CtrlEntID, "SENDRESPONSE", RequestID )

SETRESPONSECONTENT method

Sets the content to return to the client, such as a string containing an HTML page.

 SetOK= Exec_Method( CtrlEntID, "SETRESPONSECONTENT", RequestID, |
                                                      Content )

SETRESPONSECOOKIE method

Sets a cookie that is returned to the client via the “Set-Cookie” header.

 SetOK = Exec_Method( CtrlEntID, "SETRESPONSECOOKIE", RequestID,  |
                                                      CookieName, |
                                                      CookieValue )

CookieValue is an @fm-delimited array formatted as follows:

    <1> Value
    <2> Path
    <3> Domain
    <4> Expires (internal date format)
    <5> Max Age (seconds)
    <6> Secure  (TRUE$/FALSE$)
    <7> HttpOnly (TRUE$/FALSE$)
    <8> SameSite

SETRESPONSEFILE method

If you have a file that contains the content you wish to return then you should use this method to let the server read the file and return it to the client itself. This offers better performance than reading the contents via Basic+ and using the SETRESPONSECONTENT method as it avoids any unnecessary copying of data.

 SetOK = Exec_Method( CtrlEntID, "SETRESPONSEFILE", RequestID,  |
                                                    FileNamePath )

SETRESPONSEHEADER method

Sets a response header and value to be returned to the client.

 SetOK = Exec_Method( CtrlEntID, "SETRESPONSEHEADER", RequestID,  |
                                                      HeaderName, |
                                                      HeaderValue )

SETRESPONSESTATUS method

Sets the HTTP status code for the response (200, 404, 500 etc).

 SetOK = Exec_Method( CtrlEntID, "SETRESPONSESTATUS", RequestID, |
                                                      StatusCode )

Example HTTPREQUEST handler

As part of version 10.2 we have included a sample HTTPREQUEST event handler called RTI_RUN_HTTPSERVER_REQUEST which you can examine and copy for your own applications if you wish. It emulates the core behavior of the OECGI RUN_INET_REQUEST handler in that it uses the “PathInfo” field to determine the stored procedure to fulfill the request. In this case it looks for a matching procedure that has the prefix “HTTPSVR_” and we have included a couple of example “HTTPSVR_” procedures for you to review as well.

Conclusion

With the addition of the HTTPSERVER control it is now possible to provide HTML content directly from your application, and also provide a means of web-development directly from your desktop without necessarily needing to install a dedicated web-server like IIS.

It is also a good solution for when you want to provide local HTML content in your application’s user-interface via an embedded browser control, because it can avoid the usual security restrictions that browsers enforce for such scenarios.

String comparison in OpenInsight – Part 4 – String-Only operators

1 Reply

Basic+ is a dynamically-typed language, and sometimes the system will attempt to change the type of a variable before using it in an operation. For example, if we have variable containing a numeric value as a string, and we attempt to compare it to a variable containing an actual numeric value, then the former will be coerced to a numeric format before the comparison, e.g.:

   StrVar = "3"  ; // String representing an integer
   NumVar = 7    ; // Integer

   Ans = ( StrVar = NumVar ) ; // Ans is FALSE$ and StrVar 
                             ; // is converted to an Integer

This works well in the vast majority of cases, but causes issues when attempting to compare strings that that could be numbers but really aren’t for the purposes of the comparison. For example, consider the following:

   Var1 = "12"    ; // String
   Var2 = "012"   ; // String

   Ans = ( VarA = VarB ) ; // Ans = TRUE$ - should be FALSE$!!

The system first attempts to coerce the operands to a numeric format, and of course it can because they both evaluate to the number 12, and are therefore treated as equal. If you wish to compare them both as strings, the usual solution has been to change them so that they cannot be converted to a number, usually by prefixing them with a letter or symbol like so:

   Var1 = "12"    ; // String
   Var2 = "012"   ; // String

   Ans = ( ( "X" : VarA ) = ( "X" : VarB ) ) ; // Ans = FALSE$

Although this works it hurts performance and can be a cumbersome solution.

OpenInsight 10.2 introduces a new set of extended comparison operators to Basic+ that ensure both operands are coerced to a string type (if needed) before being evaluated, and provides the following benefits:

Removes the need for any concatenation thereby avoiding unnecessary copying
Removes the need to check if a variable can be a number before the comparison
Makes it obvious that you are performing a “string-only” comparison so your code looks cleaner.

The new (case-sensitive) operators are:

Operator	Description
_eqs	Equals operator
_nes	Not Equals operator
_lts	Less Than operator
_les	Less Than Or Equals operator
_gts	Greater Than operator
_ges	Greater Than Or Equals operator

There is also matching set of case-insensitive operators too:

Operator	Description
_eqsc	Case-Insensitive Equals operator
_nesc	Case-Insensitive Not Equals operator
_ltsc	Case-Insensitive Less Than operator
_lesc	Case-Insensitive Less Than Or Equals operator
_gtsc	Case-Insensitive Greater Than operator
_gesc	Case-Insensitive Greater Than Or Equals operator

So we could write the previous example like so:

   Var1 = "12"    ; // String
   Var2 = "012"   ; // String

   Ans = ( VarA _eqs VarB ) ; // Ans = FALSE$

Hopefully you’ll find this a useful addition when performing string-only comparisons in your own applications.

EditTables – Getting multi-select data the easy way

Leave a reply

In previous versions of OpenInsight the usual way of accessing data from a multi-row select EditTable control was to get the SELPOS property and then iterate over the data pulling out the rows, e.g. something like this (not optimized, but you get the idea):

   SelPos   = Get_Property( CtrlEntID, "SELPOS" )
   DataList = Get_Property( CtrlEntID, "LIST" )
   SelList  = ""

   SelRows  = SelPos<2>
   SelCount = FieldCount( SelRows, @Vm )  
   For SelIdx = 1 To SelCount
      SelRow = SelRows<0,SelIdx>
      SelList<-1> = DataList<SelRow>
   Next

However, in OpenInsight 10 we added a couple of new properties that allow you to access data in a multi-row select EditTable in a faster and more efficient way. These are:

The SELLIST property
The SELARRAY property

Both of these return data in the familiar LIST and ARRAY formats, but they only return data from the selected rows, thereby saving you the step of accessing SELPOS and iterating over the data yourself. So, to rewrite the example above we can now do this:

   SelList = Get_Property( CtrlEntID, "SELLIST" )

Likewise, to return the data in ARRAY format we would use the SELARRAY property like so:

   SelArray = Get_Property( CtrlEntID, "SELARRAY" )

Ergo, when asked the other day “What’s the fastest way of getting data from a specific column from the selected rows”, the answer was:

   SelData = Get_Property( CtrlEntID, "SELARRAY" )<colNum>

(And this question is also what led me to write this post…)

Bonus Trivia

The LISTBOX control also supports the SELLIST property.

The DIRWATCHER control

Leave a reply

With the release of version 10.1 a new control type called DIRWATCHER (“Directory Watcher”) has been added to OpenInsight. This is a fairly simple control which allows you to monitor one or more directories on your system and then receive notifications when the contents are changed.

Using the control is very straightforward:

Use the WATCHDIR method to add a directory to monitor for changes.
Handle the CHANGED event to receive notifications of directory changes.
Use the STOP method to stop monitoring directories.

We’ll take a quick look at each of these methods and events below along with a couple of important properties:

The WATCHDIR method

This method allows you to specify a directory to monitor along with some optional flags. It may be called multiple times to watch more than one directory.

bSuccess = Exec_Method( CtrlEntID, "WATCHDIR", DirName, bSubtree, Flags )

Parameter	Description
DirName	(required) Specifies the name of the directory to watch. It should be a fully qualified, non-relative directory path.
bSubtree	(optional) Set to TRUE$ to monitor all sub-directories beneath DirName as well. Defaults to FALSE$.
Flags	(optional) A bit-mask value containing a set of flags that denote the events to monitor in the directories. It defaults to the following flags: FILE_NOTIFY_CHANGE_FILE_NAME$ FILE_NOTIFY_CHANGE_LAST_WRITE$ FILE_NOTIFY_CHANGE_CREATION$ The flag values are specified in the MSWIN_FILENOTIFY_EQUATES insert record,

This method returns TRUE$ if successful, or FALSE$ otherwise.

The STOP method

This method stops the control monitoring its specified directories.

bSuccess = Exec_Method( CtrlEntID, "STOP" )

This method returns TRUE$ if successful, or FALSE$ otherwise.

(Note – to resume directory monitoring after the STOP method has been called the WATCHDIR method(s) must be executed again).

The CHANGED event

This event is raised when changes have been detected in the monitored directories.

bForward = CHANGED( NewData )

This event passes a single parameter called NewData which contains an @vm-delimited list of changed items (i.e. notifications). Each item in the list comprises an “action code” and the name and path of the affected file, delimited by an @svm.

Action codes are defined in the MSWIN_FILENOTIFY_EQUATES insert record like so:

   equ FILE_ACTION_ADDED$               to 0x00000001   
   equ FILE_ACTION_REMOVED$             to 0x00000002   
   equ FILE_ACTION_MODIFIED$            to 0x00000003   
   equ FILE_ACTION_RENAMED_OLD_NAME$    to 0x00000004   
   equ FILE_ACTION_RENAMED_NEW_NAME$    to 0x00000005

Remarks

If a monitored directory experiences a high volume of changes (such as copying or removing thousands of files) it could generate a correspondingly high number of CHANGED events, which in turn could produce an adverse affect on your application and slow it down. In order to deal with this potential issue it is possible to “bundle up” multiple notifications with the NOTIFYTHRESHOLD property into a single CHANGED event so they may be processed more efficiently.

The NOTIFYTHRESHOLD property

The NOTIFYTHRESHOLD property is an integer that specifies the maximum number of notifications that should be bundled before a CHANGED event is raised.

CurrVal = Get_Property( CtrlEntID, "NOTIFYTHRESHOLD" )
PrevVal = Set_Property( CtrlEntID, "NOTIFYTHRESHOLD", NewVal )

By default it is set to 100.

The NOTIFYTIMER property

The NOTIFYTIMER property is an integer that specifies the number of milliseconds before a CHANGED event is raised if the NOTIFYTHRESHOLD property value is not met.

CurrVal = Get_Property( CtrlEntID, "NOTIFYTIMER" )
PrevVal = Set_Property( CtrlEntID, "NOTIFYTIMER", NewVal )

By default it is set to 50 (ms).

Remarks

If the NOTIFYTHRESHOLD property is set to a value greater than 1 then the control will try to bundle that number of notifications together before raising a CHANGED event. However, when this is set to a high value it is possible that the threshold may not be reached in a timely fashion and the CHANGED event not actually raised.

E.g. If the NOTIFYTHRESHOLD is set to 1000, and only 200 notifications are received then the CHANGED event would not be raised.

To prevent this problem the NOTIFYTIMER property may be used to specify the amount of time after receiving the last notification before a CHANGED event is raised even if the NOTIFYTHRESHOLD is not met.

E.g. in the example above, if the control had a NOTIFYTIMER of 50, then a CHANGED event would be raised 50ms after the last notification (200) was received, even though the NOTIFYTHRESHOLD of 1000 has not actually been met.

Developer Notes

The DIRWATCHER control is intended as a “non-visual” control and should probably be hidden at runtime in your own applications. However, it is actually derived from a normal STATIC control so all of the properties and methods that apply to a STATIC apply to the DIRWATCHER as well, and you may use them as normal if you wish.

Yield or Die!

1 Reply

When executing a long running process in a desktop environment, such as selecting records from a large table, it is important for the user to be able to interact with the application even though it is busy. A common example of this is displaying a dialog box that shows the progress of an operation and allowing the user to cancel it if they wish. Failure to provide this ability generally results in user frustration, or causes the dreaded “Window Ghosting” effect where Windows changes a form’s caption to “Not Responding” (this is never a good look, and usually ends in a quick visit to the Task Manager to kill the application).

In order to avoid this problem we have to allow the application to check for user interaction, a process usually referred to as “yielding” (hence the awful title of this post), and this time we’ll take a look at the various options available to accomplish this and the differences between them. Before we go any further however, here’s a little background information on how OpenInsight runs beneath the hood so that you can appreciate how messages and events are handled.

Under the hood

OpenInsight.exe (aka. the “Presentation Server”, or “PS”) has a main thread (the “UI thread”) with a Windows message loop that manages all of the forms and controls, and it also has an internal “event queue” for storing Basic+ events that need to be executed. The PS also creates an instance of the RevEngine virtual machine (“the engine”), which has its own thread (the “engine thread”) with a Windows message loop, and is responsible for executing Basic+ code.

When the PS needs to execute an event it passes it to the engine directly if possible, otherwise it adds the details to the event queue and then posts a message to itself so the queue can be checked and processed when the engine is not busy. When the engine receives the event data it is executed on the engine thread. Stored procedures such as Get_Property, Set_Property, and Exec_Method provide a way for the Basic+ event to communicate back to the PS to interact with the user interface controls and forms during its execution.

The key point to note here is that Basic+ event code runs in a different thread to the UI, so while the engine thread is processing the event, the UI thread is basically waiting for it to finish, and this means that it may or may not get chance to process it’s own message loop. This is where the problems can begin, and why the need for a yielding ability, because:

The engine thread needs to be paused or interrupted in some fashion so that the UI thread can check and process its own Windows message queue for things like mouse, keyboard and paint messages. If this queue is not checked at least every 10 seconds then Windows assumes that the PS is hung and the “Not Responding” captions are shown on the application forms.
While the engine is processing an event, the PS cannot pass it a new one, so it is added to the event queue. If we are waiting to process some Basic+ event code like a button CLICK to cancel the current operation, then we need some way for this to be retrieved and executed before the current event is finished.

So, now we know why “window ghosting” happens we can take a look at the various options to deal with it.

Options for yielding

MSWin_Sleep stored procedure

This is a direct call to the Windows API Sleep function, and it puts the engine thread to sleep for at least the specified number of milliseconds. However, while calling this will allow Windows to schedule another thread to run, there’s no guarantee that this would be the UI thread, so it’s not really a good solution.

WinYield stored procedure

This is a simple wrapper around the Windows API Sleep function, with a sleep-time of 10ms. This suffers from the same disadvantages discussed for MSWin_Sleep above (This function remains for backwards compatibility with early versions of of OI and Windows).

MSWin_SwitchToThread stored procedure

This is a direct call to the Windows API SwitchToThread function which forces Windows to schedule another thread for execution. Like MSWin_Sleep and WinYield there’s no guarantee that this would cause the UI thread to run, so again it’s not a great solution.

SYSTEM PROCESSEVENTS method (a.k.a Yield stored procedure)

This is a new method in version 10.1 that performs two tasks that solve the problem:

It explicitly tells the UI thread to process its message queue (which will avoid the “ghosting” issue), and
It allows the UI thread to process the event queue so waiting events can be executed as well.

One possible drawback here is that waiting events are also processed, and this might not be a desirable outcome depending on what you are doing. In this case there is another method called PROCESSWINMGS that should be used instead.

(FYI – The PROCESSEVENTS method is essentially a wrapper around the venerable Yield() stored procedure, but allows the same functionality to be called via the standard object-based method API rather than as a “flat” function. Yield() itself is still and will be fully supported in the product).

SYSTEM PROCESSWINMSGS method

This is a new method in version 10.1 that tells the PS to process it’s Windows message queue but it does not process any Basic+ events, i.e. it prevents the “ghosting” effect but does not cause events to fire before your application is ready for them.

Conclusion

Version 10.1 has added more functionality to help you avoid the dreaded “Not Responding” message via the PROCESSWINMGS and PROCESSEVENTS methods, and hopefully, armed with the information above, this will help you to write better integrated desktop-applications.

The Case of the Extra Pixel

2 Replies

In the current OpenInsight 10.1 Beta program, Martyn at RevSoft UK had reported a bug about the Height property creeping up by a pixel each time he opened a form. This was traced to the point when the menu structure was parsed and created: it was actually inserting an extra pixel, so this was fixed and the form Height now stayed the same between each opening.

However, a little more testing against a form without a menu revealed another issue – in some cases a pixel was added to the Height but it didn’t creep up on each subsequent opening:

E.g. when set to a Height of 400 and saved, the form would re-open with a Height of 401, but it would stay at the same value afterwards; not the same bug as before but it did need investigating…

The Height and the ClientHeight properties

As some of you will know, Windows forms are split into two areas ⁽¹⁾:

The “Nonclient area” which contains items such as the borders, menus and title bars, and
The “Client area”, which is the part that contains the controls.

When an OpenInsight form definition is saved the actual Width and Height properties are not used. Instead, the ClientWidth and ClientHeight properties (i.e. the dimensions of the Client area) are saved because Windows can change the size of the Nonclient parts when a form is run on different systems with different visual styles, and this can make the form’s Client area the wrong size when created from a saved Width and Height (as we all found out many years ago when Windows XP was released). In our particular case above, when the form was saved and reopened, the ClientWidth and ClientHeight properties were correct, but the Height had gained a pixel.

E.g. When set to a Height of 400, the saved form ClientHeight was 365. When the same form was reopened the ClientHeight was still 365, but the Height was now reported as 401.

Height, ClientHeight and High DPI

I run my primary and secondary monitors at different DPI settings to ensure that scaling works correctly, and in this case, at 192 DPI (i.e. scaled to 200%), it transpired that the integer rounding used during scaling was the issue because:

Setting the Height to 400 resulted in a ClientHeight of 365.
Setting the Height to 401 also resulted in a ClientHeight of 365.
Setting the ClientHeight to 365 resulted in a Height of 401.

I.e. setting the ClientHeight to a specific value, and then retrieving the form’s actual height in real pixels, and then scaling it back to 96 DPI (all values in the Form designer are shown and stored at 96 DPI), gave the extra pixel. Because we don’t record the Height in the form definition we have no way of knowing that the ClientHeight was set from a value of 400 rather than 401 when the form was reopened in the designer, so we have to go with the 401. Mystery solved!

Of course, this looks odd, but it’s just an artifact of the scaling calculations. The crucial value is the ClientHeight because this is the value that is recorded and used, and this is what needs to be preserved when forms are saved and reopened. To help put your mind at ease about this, the ClientWidth and ClientHeight properties have now been exposed (for forms only) in the Form Designer, so you can be confident that the correct size is always being maintained (ClientWidth and ClientHeight are normally runtime only properties).

E.g. In the following two images (saved and reopened) you can see that the pixel height has increased, but in both cases the ClientHeight (365) is preserved and is correct:

Image of form with a saved Height property of 400 — Form saved with a Height of 400

Image of reopened form with a Height property of 401 — Form reopened, now with a Height of 401

Conclusion

Windows XP taught us many years ago that the Width and Height properties are not reliable when creating a forms as they can produce inconsistent results on different systems, so we always rely on the ClientWidth and ClientHeight properties instead.
Don’t be concerned if you see a slightly different Height value when you reopen a form if you’re running at a high DPI setting – the crucial property is the ClientHeight value – as long as this is consistent there is no actual problem.
To make sure you can monitor this yourself the ClientWidth and ClientHeight properties have been exposed in the Form Designer, and you can edit these directly if you wish.

(Note: the ClientHeight and ClientWidth properties are only exposed after builds later than the current Beta 3 release)

⁽¹⁾ If you are not familiar with Client and Nonclient areas in Windows GUI programming you can find out more information here).