Monthly Archives: September 2022

Migration tool updates in OpenInsight 10.2

Based on user feedback (and our own experience) version 10.2 includes the following new updates for the OpenInsight migration process that we think you’ll find invaluable when moving your applications:

The ability to choose individual migration tasks
The ability to select individual entities
The ability to migrate SYSPROG components

We’ll take a look at each of these features below, along with a small note on “migrating” your data.

Task Selection

In previous releases the Migration Tool had a fixed set of tasks to perform which it did in sequence. In this release you may now specify which tasks you wish to execute instead, which might save some time if you have to repeat the migration process more than once (a common requirement if needing to keep a version 9 application synchronized with version 10 during upgrade development).

By default all tasks are selected, but you may deselect any that you don’t wish to run.

Task	Description
Migrate App Settings	Migrates information from the SYSAPPS record. Migrates database environment settings. Migrates custom event (CFG_EVENTS) settings.
Migrate Users	Migrates user details into the SYSUSERS table.
Migrate DLL Prototypes	Migrates “DLL_” records in SYSPROCS to DLLPROTOTYPE entities.
Migrate C-Structures	Migrates “STRUCT_” records in SYSOBJ to DLLSTRUCT entities.
Migrate Promoted Events	Migrates promoted event records in SYSREPOSEVENTS/EXES to PROMOTEDEVENT/EXE entities.
Migrate RDK records	Migrates SYSREPOSVIEWS/RELEASES records to REPVIEW/RELEASE entities.
Migrate Entities	Migrates application entities. Note that these can now be specified individually if desired (see below).
Migrate Arev32 records	Migrates Arev32-specific records in SYSENV. Migrates Arev32 records from SYSPRINTERS.
Clean Relationships	Removes invalid relationships (Using/Used By).
Compile Stored Procedures	Compiles migrated stored procedures.
Compile Promoted Events	Compiles migrated promoted events.
Compile Forms	Compiles migrated forms.

Migration Tool Tasks

Entity Selection

If you choose the “Migrate Entities” task then you now have the option to select individual entities to migrate (If you don’t select any then all entities in the v9 application will be migrated).

To select individual entities right-click on the “Selected Entities” control and choose “Select Entities” from the menu, or click the “…” options button in the top-right of the control’s header:

You may now use the “Select Repository Entities” dialog to choose which entities you wish to migrate:

As you can see there are quite a few options you can use to find the entities you wish to migrate – these options make a “repository filter” that is used to create a list to choose from:

Option	Description
Filter ID	Name of a saved filter (REPFILTER entity). Choosing a filter will load the other selection criteria controls in this dialog with the saved filter details.
View ID	Name of a saved Repository View (REPVIEW entity). If specified, the list of entities in this View is used as a starting point for the filter when it executes.
Entity ID	If specified only entities matching this name will be selected. The standard ‘[]’,’]’ and ‘[‘ RList-style syntax for “containing”, “starting with” and “ending with” is respected.
Date From	If specified only entities updated from this date will be included. This option supports a “nD” syntax where “n” is the number of days since the current date. E.g. to select entities updated since the previous day specify “1D”, for entities updated in the last week specify “7D” and so on.
Date To	If specified do not include entities after this date. This option supports the same “nD” syntax as the “Date From” option.
Updated By	If specified then only entities updated by the specified user(s) are returned. This option may contain a “,” delimited list for matching against multiple users.
Filter Options	Only entities matching the checked options will be returned. Note that for a migration operation the “Show Inherited Entities” checkbox is always disabled (This dialog is used elsewhere in the system to select repository entities, so it can be enabled in such scenarios).
Filter By Types	Allows you to choose specific entity types and classes to include. If blank then all types and classes are included in the search. Click the “…” options button in the top right corner of the control, or right-click to use the context menu to launch a dialog to select the types and classes.
Filter By SELECT	This option allows you to enter an RLIST SELECT statement against the SYSREPOS table that will be used to filter the entities.
Save Filter	Click this button to save the filter criteria for future re-use (stored as a REPFILTER entity in the repository).
Apply Filter	Click this button to execute the filter criteria and load a list of matching entities into the “Available” controls below. If you have not entered any criteria then all entities in the v9 application will be selected.

Filter Options

Once you have applied your filter the “Available Types” control is populated – selecting a Type populates the “Available Entities” control where you can select the items that you wish to migrate. Use the “>” and “>>” buttons to add them to the “Selected Entities” control on the right, or the “<” and “<<” buttons to remove them.

Things to note:

As items in the “Available Entities” control are selected, the list of items that they use is shown in the “Related (Used By) Entities” control below. These can also be selected if you wish.
Each of the “Available” and “Selected” controls have context menus and “…” options buttons (top-right corner).
If the “Auto-Add Executable” option is checked when you add entities to the “Selected Entities” control then any executable versions of the entity (e.g. STPROCEXE, OIWINEXE etc) are also added.
If the “Auto-Add Source” option is checked when you add entities to the “Selected Entities” control then any source versions of an executable entity (e.g. STPROC, OIWIN etc) are also added.

Clicking “OK” takes you back to the main migration form where you will see the list of entities selected. From there you may click the “Next” button to start the migration process.

(Note that the task and entity selections are saved automatically between migrations so that you may use them again if you wish.)

Migrating items from SYSPROG

Previous versions of the migration tool prevented migrating any entities from the SYSPROG application. This was to preserve the system from being damaged in case older v9 entities overwrote the v10 ones, thereby rendering it unusable.

With this release you may run the migration form from your SYSPROG application with the following restrictions:

You may only execute the following tasks:
- Migrate RDK Records
- Migrate Entities – You must specify the entities to migrate – you cannot migrate “All Entities”!
- Migrate Arev32 Records
- Clean Relationships
- Compile Stored Procedures
- Compile Forms

Bonus Trivia – a quick note on migrating data

The migration process begins with creating a new application in version 10 and then executing a form called RTI_MIGRATE_V9_TO_V10, which guides you through the steps necessary to move your application from version 9.

At this point you select the location of the application you are migrating from, after which you see the data volumes that are attached in it. There are options for the data to be “migrated”, but in my experience it is best to move and attach the data you want for your application before you attempt the migration rather than use the facility here. The migration process does not change your data at all so there is no real advantage to using this facility (it is basically provided as a convenience), and in fact, if you are moving tables containing relational indexes they will be removed. It is far easier to copy/move and attach the tables yourself using the Database ToolPanel before you get here.

More Bonus Trivia – Sharing index data between v9 and v10

While on the topic of migrating data it is also worth taking a look at the end of this blog post by Sprezzatura if you intend to share your tables between v9 and v10 systems – you must make sure that you run the indexing system in a v9 compatible format:

Indexing Issues In Large Tables

Conclusion

The migration tool has been significantly improved for the version 10.2 release and we’re sure you will find it much more helpful when moving your older applications to the latest version of OpenInsight.

ListBoxes and TreeListBoxes – “in-place” editing

The HTTPSERVER control

2 Replies

OpenInsight 10.2 introduces a new control type called HTTPSERVER, which provides a lightweight HTTP web-server control for use in your applications.

Using the control is a simple process:

Drop a “HTTP Server” control onto a form
Set the port number to listen on via the PORT property
Use the START method to start the server, or set the STARTUPMODE property to “Automatic” if you want the server to start when the form is created.
Listen for requests via the HTTPREQUEST event and respond to them using methods such as the SETRESPONSECONTENT and SETRESPONSEFILE methods
Return the response using the SENDRESPONSE method

So, if you set up a HTTPSERVER on port 8089 and execute the form you can use a browser to communicate with it via a URL something like this:

http://localhost:8089/path1/path2/test?arg1=this&arg2=that

The HTTPREQUEST event

As you can see, setting up the control is fairly easy, but the bulk of the work needs to be done in the HTTPREQUEST event where you examine the contents of the request and return the appropriate content.

The event has the following interface:

   bForward = HTTPREQUEST( CtrlEntID,     |
                           CtrlClassID,   |
                           RequestID,     |
                           RequestHeaders )

And the following parameters:

Name	DESCRIPTION
CtrlEntID	ID of the HTTP Server control firing the event
CtlrClassID	Type of control firing the event – this is always “HTTPSERVER”
RequestID	Unique identifier for returning a response to the client – this is used with the various “SETRESPONSE” methods that set response data.
RequestHeaders	An @FM delimited dynamic array of data that describes the request. The format is similar to the HTTPRequest argument used in OECGI programming. The full format is described in the PS_HTTPSERVER_EQUATES insert record.

As mentioned above, the RequestHeaders parameter describes the details of the request using a format similar to the format used in OECGI programming. There are some differences that are worth highlighting however:

For a GET request the query values are already parsed into their own fields (<37> and <38>) as an associated multi-value pair. They are not found unparsed in field <1> as per OECGI.
For a POST or PUT request the content is obtained using the GETREQUESTCONTENT method (see below) – it is not passed in the RequestHeaders variable.
Cookies are already parsed into their own fields (<39> and <40>) as an associated multi-value pair.
Headers are already parsed into their own fields (<35> and <36>) as an associated multi-value pair.

Note that out of the box we do not enforce any restrictions or framework on how you handle the request – compare this to classic OECGI programming where the “PathInfo” field is used to determine which “INET_” procedure is executed to fulfill it (via the RUN_INET_REQUEST stored procedure) There is no such requirement with the HTTPSERVER control, and you may create your own framework if you wish (although see the note on RTI_RUN_HTTPSERVER_REQUEST below).

Returning a response

There are several methods described below that you may use to process the content that you return to the client.

GETREQUESTCONTENT
GETRESPONSECONTENT
GETRESPONSECOOKIE
GETRESPONSEFILE
GETRESPONSEHEADER
GETRESPONSESTATUS
ISPORTINUSE
SETRESPONSECONTENT
SETRESPONSECOOKIE
SETRESPONSEFILE
SETRESPONSEHEADER
SETRESPONSESTATUS
SENDRESPONSE

Note: With each of these you must use the unique RequestID parameter passed to you in the HTTPREQUEST event.

E.g. Returning HTML in the HTTPREQUEST event

ReposID = @AppID<1> : "*DOC*HTML*INDEX_PAGE" 
HTML = Repository( "ACCESS", ReposID )
Call Exec_Method( CtrlEntID, "SETRESPONSECONTENT", RequestID, HTML )
Call Exec_Method( CtrlEntID, "SENDRESPONSE" )

E.g. Returning a file in the HTTPREQUEST event

ReposID = @AppID<1> : "*DOC*HTML*INDEX_PAGE" 
FilePath = Repository( "GETSUBKEY", ReposID )
Call Exec_Method( CtrlEntID, "SETRESPONSEFILE", RequestID, FilePath )
Call Exec_Method( CtrlEntID, "SENDRESPONSE" )

The HTTPSERVER methods

GETREQUESTCONTENT method

Gets the raw content sent to the server by the client as part of a POST or PUT request.

 ReqContent = Exec_Method( CtrlEntID, "GETREQUESTCONTENT", RequestID )

GETRESPONSECONTENT method

Returns raw content for the response as set via a previous call to the SETRESPONSECONTENT method.

 RspContent= Exec_Method( CtrlEntID, "GETRESPONSECONTENT", RequestID )

GETRESPONSECOOKIE method

Returns details for a response cookie as set via a previous call to the SETRESPONSECOOKIE method (see below for the CookieVal format).

 CookieVal = Exec_Method( CtrlEntID, "GETRESPONSECOOKIE", RequestID, |
                                                          CookieName )

GETRESPONSEFILE method

Returns the name and path of the response content file set with a previous call to the SETRESPONSEFILE method.

 RspFile = Exec_Method( CtrlEntID, "GETRESPONSEFILE", RequestID )

GETRESPONSEHEADER method

Returns the details for a response header as set via a previous call to the SETRESPONSEHEADER method.

 HeaderVal = Exec_Method( CtrlEntID, "GETRESPONSEHEADER", RequestID, |
                                                          HeaderName )

GETRESPONSESTATUS method

Returns the HTTP status code of the response (defaults to 200).

 RspStatus = Exec_Method( CtrlEntID, "GETRESPONSESTATUS", RequestID )

ISPORTINUSE method

Returns TRUE$ if the specified port is in use on the local machine.

 InUse = Exec_Method( CtrlEntID, "ISPORTINUSE", Port, IPv6 )

Port – Identifies the port to check
IPv6 – if TRUE$ then check the IPv6 bindings, otherwise check the IPv4 bindings

SENDRESPONSE method

Sends the response back to the client. This method should be called when you have finished setting the response details (Note that this is called by the promoted system HTTPREQUEST handler in case you forgot to do it in your own code!).

 RspSent = Exec_Method( CtrlEntID, "SENDRESPONSE", RequestID )

SETRESPONSECONTENT method

Sets the content to return to the client, such as a string containing an HTML page.

 SetOK= Exec_Method( CtrlEntID, "SETRESPONSECONTENT", RequestID, |
                                                      Content )

SETRESPONSECOOKIE method

Sets a cookie that is returned to the client via the “Set-Cookie” header.

 SetOK = Exec_Method( CtrlEntID, "SETRESPONSECOOKIE", RequestID,  |
                                                      CookieName, |
                                                      CookieValue )

CookieValue is an @fm-delimited array formatted as follows:

    <1> Value
    <2> Path
    <3> Domain
    <4> Expires (internal date format)
    <5> Max Age (seconds)
    <6> Secure  (TRUE$/FALSE$)
    <7> HttpOnly (TRUE$/FALSE$)
    <8> SameSite

SETRESPONSEFILE method

If you have a file that contains the content you wish to return then you should use this method to let the server read the file and return it to the client itself. This offers better performance than reading the contents via Basic+ and using the SETRESPONSECONTENT method as it avoids any unnecessary copying of data.

 SetOK = Exec_Method( CtrlEntID, "SETRESPONSEFILE", RequestID,  |
                                                    FileNamePath )

SETRESPONSEHEADER method

Sets a response header and value to be returned to the client.

 SetOK = Exec_Method( CtrlEntID, "SETRESPONSEHEADER", RequestID,  |
                                                      HeaderName, |
                                                      HeaderValue )

SETRESPONSESTATUS method

Sets the HTTP status code for the response (200, 404, 500 etc).

 SetOK = Exec_Method( CtrlEntID, "SETRESPONSESTATUS", RequestID, |
                                                      StatusCode )

Example HTTPREQUEST handler

As part of version 10.2 we have included a sample HTTPREQUEST event handler called RTI_RUN_HTTPSERVER_REQUEST which you can examine and copy for your own applications if you wish. It emulates the core behavior of the OECGI RUN_INET_REQUEST handler in that it uses the “PathInfo” field to determine the stored procedure to fulfill the request. In this case it looks for a matching procedure that has the prefix “HTTPSVR_” and we have included a couple of example “HTTPSVR_” procedures for you to review as well.

Conclusion

With the addition of the HTTPSERVER control it is now possible to provide HTML content directly from your application, and also provide a means of web-development directly from your desktop without necessarily needing to install a dedicated web-server like IIS.

It is also a good solution for when you want to provide local HTML content in your application’s user-interface via an embedded browser control, because it can avoid the usual security restrictions that browsers enforce for such scenarios.

String comparison in OpenInsight – Part 4 – String-Only operators

1 Reply

Basic+ is a dynamically-typed language, and sometimes the system will attempt to change the type of a variable before using it in an operation. For example, if we have variable containing a numeric value as a string, and we attempt to compare it to a variable containing an actual numeric value, then the former will be coerced to a numeric format before the comparison, e.g.:

   StrVar = "3"  ; // String representing an integer
   NumVar = 7    ; // Integer

   Ans = ( StrVar = NumVar ) ; // Ans is FALSE$ and StrVar 
                             ; // is converted to an Integer

This works well in the vast majority of cases, but causes issues when attempting to compare strings that that could be numbers but really aren’t for the purposes of the comparison. For example, consider the following:

   Var1 = "12"    ; // String
   Var2 = "012"   ; // String

   Ans = ( VarA = VarB ) ; // Ans = TRUE$ - should be FALSE$!!

The system first attempts to coerce the operands to a numeric format, and of course it can because they both evaluate to the number 12, and are therefore treated as equal. If you wish to compare them both as strings, the usual solution has been to change them so that they cannot be converted to a number, usually by prefixing them with a letter or symbol like so:

   Var1 = "12"    ; // String
   Var2 = "012"   ; // String

   Ans = ( ( "X" : VarA ) = ( "X" : VarB ) ) ; // Ans = FALSE$

Although this works it hurts performance and can be a cumbersome solution.

OpenInsight 10.2 introduces a new set of extended comparison operators to Basic+ that ensure both operands are coerced to a string type (if needed) before being evaluated, and provides the following benefits:

Removes the need for any concatenation thereby avoiding unnecessary copying
Removes the need to check if a variable can be a number before the comparison
Makes it obvious that you are performing a “string-only” comparison so your code looks cleaner.

The new (case-sensitive) operators are:

Operator	Description
_eqs	Equals operator
_nes	Not Equals operator
_lts	Less Than operator
_les	Less Than Or Equals operator
_gts	Greater Than operator
_ges	Greater Than Or Equals operator

There is also matching set of case-insensitive operators too:

Operator	Description
_eqsc	Case-Insensitive Equals operator
_nesc	Case-Insensitive Not Equals operator
_ltsc	Case-Insensitive Less Than operator
_lesc	Case-Insensitive Less Than Or Equals operator
_gtsc	Case-Insensitive Greater Than operator
_gesc	Case-Insensitive Greater Than Or Equals operator

So we could write the previous example like so:

   Var1 = "12"    ; // String
   Var2 = "012"   ; // String

   Ans = ( VarA _eqs VarB ) ; // Ans = FALSE$

Hopefully you’ll find this a useful addition when performing string-only comparisons in your own applications.

EditTables – Getting multi-select data the easy way

Bonus Trivia

The LISTBOX control also supports the SELLIST property.

Building OpenInsight 10

News and notes from the bleeding edge of Revelation Software development

Monthly Archives: September 2022

Migration tool updates in OpenInsight 10.2

Task Selection

Entity Selection

Migrating items from SYSPROG

Bonus Trivia – a quick note on migrating data

More Bonus Trivia – Sharing index data between v9 and v10

Conclusion

ListBoxes and TreeListBoxes – “in-place” editing

The HTTPSERVER control

The HTTPREQUEST event

Returning a response

The HTTPSERVER methods

GETREQUESTCONTENT method

GETRESPONSECONTENT method

GETRESPONSECOOKIE method

GETRESPONSEFILE method

GETRESPONSEHEADER method

GETRESPONSESTATUS method

ISPORTINUSE method

SENDRESPONSE method

SETRESPONSECONTENT method

SETRESPONSECOOKIE method

SETRESPONSEFILE method

SETRESPONSEHEADER method

SETRESPONSESTATUS method

Example HTTPREQUEST handler

Conclusion

String comparison in OpenInsight – Part 4 – String-Only operators

EditTables – Getting multi-select data the easy way

Bonus Trivia