Category Archives: OpenEngine

Extended Precision Mode

One of the common problems faced by many programmers is the ensuring the accuracy of floating point arithmetic, mainly due to the rounding errors that can occur when calculations are performed on numbers that cannot be fully represented in a binary floating-point format (The size of this format determines the precision, and therefore the accuracy of calculations).

OpenInsight is built with the Microsoft C++ compiler, which limits the floating-point format to 64-bits (known as the “Double” type), and Basic+ variables that represent non-integer numbers use this type internally.  The use of this 64-bit format is one of the problems commonly noticed by developers who have moved their systems from the older Advanced Revelation platform to OpenInsight, because the internal floating-point format for R/Basic variables was the 80-bit “Long Double” type instead.  This means that calculations from the same RBasic/Basic+ code running on ARev and OI may produce different results due to this reduction in precision.

(The Long Double is unfortunately not supported by the MS C++ compiler as, according to the VC++ compiler team back in 2006: “The major reason is that FP code generation has been switching to the use of SSE/SSE2/SSE3 instruction sets instead of the x87 FP stack since that is what both the AMD and Intel recent and future chip generations are focusing their performance efforts on. These instruction sets only support 32 and 64 bit FP formats”).

The “integer workaround”

A common workaround for precision problems is to control the calculations at each step so you are effectively dealing with integer operations – this can be done by using the “MD” IConv/OConv functions to control the precision, or by simply multiplying values by a known factor and dividing the result again afterwards.

Both of these methods can make the code messy and obscure the intent, and, depending on the values used, may result in integer overflow if they are too large (though this is less likely to happen on a 64-bit system like OpenInsight 10 however).

The “Extended Precision Operators”

Another option to help mitigate these calculation problems are the Extended Precision Operators that were introduced in OpenInsight 9.3:

  • _addx
  • _subx
  • _mulx
  • _divx

These do allow you to specify the precision to use at each step, but suffer from the need to rewrite existing code to use them. It is also easy to lose precision if you inadvertently use a “normal” operator in between them as well, as the following example demonstrates:

a = _divx( c, b, 24 )   ; // "a" is full precision (24)
if ( a > 1 ) then       ; // "a" converted to "double" by ">" operator
   // do stuff
end
b = _addx( a, z )       ; // "a" is no longer full precision passed to _addx

The new “Extended Precision Mode”

With the upcoming release of version 10.0.7, OpenInsight now supports a new feature for dealing with high precision calculations called Extended Precision Mode (EP Mode). When this mode is enabled all the normal maths operators switch into an “extended mode” and the results of calculations are stored in Basic+ variables using a new internal type introduced specifically for maintaining the precision.

This means that existing code can be reused by simply adding statements to activate and deactivate the mode as needed via the new SetEPMode() function , e.g:

Compile Function Mickey_Mouse_EPM_Test( void )

   Declare Function GetEPMode
   $Insert Logical
 
   epMode = GetEPMode()

   Call SetEPMode( FALSE$ )
   GoSub runTest
   
   Call SetEPMode( TRUE$ )
   GoSub runTest
   
   Call SetEPMode( epMode )

Return

runTest:
   a = 10.12346 * (22/7); 
   b = 100000; 
   For x = 1 To 1000 
      b -= a 
   Next
   Call Send_Dyn( b ) 
Return

Output:

(Normal) 68183.4114285739 
(EPMode) 68183.41142857142857142857142857143725

The following operators are affected by the EP Mode:

+
+=
-
-=
*
/
== or =
!= or <>
>
<
>=
<=
mod()
int()
abs()
atan()
cos()
exp()
ln()
pwr()
sin()
sqrt()
tan()

Performance considerations

By default EP Mode is not enabled because calculations are slower due to the extra processing needed to maintain precision, and this would have a detrimental effect on the performance of your applications if it were permanently enabled, so you should only use it when absolutely required.

You should also note that the EP Mode and Precision are set to their default values for each new request made to the engine (i.e each event or web-request).  This is to protect against situations where an error condition could force the engine to abort processing before the EP Mode settings could be reset, thereby leaving it in an undesired state (This is similar to the way UTF-8 mode works so that data integrity is preserved).

Controlling the precision level

The actual level of precision (i.e. the decimal places) is controlled by another setting, which is updated by the new SetEPModePrecision() function (The greater the precision the longer calculations will take to perform).   By default the precision is set to 32.

   decPlaces = GetEPModePrecision()
   Call SetEPModePrecision( 24 )

 Default EP settings

The default settings can be changed in the Application Properties dialog launched from the IDE Settings menu.

Further reading on floating point arithmetic

More information on floating point arithmetic can be found by following the links below:

OpenInsight 10 and the Visual C++ 2015 Runtime

As we’re aiming to update the core OpenEngine code to be a cross-platform solution we spent some of last week moving the OpenInsight 10 C++ codebase from Visual Studio 2010 to Visual Studio 2015, to take advantage of better C++ standards support and the improved tools, such as the excellent C++ static code analyzer.

All in all it was a fairly smooth process, apart from a “hidden” default compiler setting that managed to temporarily break the High DPI and scaling code, but that sort of thing is expected when moving versions, so after a fair bit of Google-Fu we found the issue and fixed it.  One big difference we did find however, is how Microsoft have changed the way their C-runtime libraries are distributed, and it’s something of a departure from previous versions.

As you may or may not be aware, programs built with Visual C++ rely on a set of core DLLs, usually known as the “VC++ Redistributable” and this can be deployed in one of two ways:

  1. The DLLs can be included in the application directory – this is the “App-folder” method,
  2. The DLLs can be installed into the system directories via a “VC++ Redistributable” setup program, which is basically another client-install program.

In order to save you the extra pain associated with yet another client-install  we opted to use the App-Folder method for OpenInsight 10, as originally this meant we supplied you with just two extra DLLs.  However, for Visual Studio 2015 Microsoft have decided to do some refactoring to share their C-runtime code between Desktop apps and the new Universal Platform apps, and so at first glance it looked we needed to supply four DLLs instead, these being:

  • concrt140.dll
  • msvcp140.dll
  • vccorlib140.dll
  • vcruntime140.dll

But unfortunately this was not the end of the story, as early testing on some machines showed that there were still some required DLLs missing.  As it turns out these four DLLs rely on a further set of DLLs that Microsoft say are now “part of the operating system”, and although we could supply these too there are actually 41 of them!  Needless to say we won’t be cluttering up your OpenInsight folders with these.

On further investigation it appears that Microsoft are indeed releasing this 41-DLL set through Windows Update, but only as an Optional Update, so its quite possible that systems will not have these installed and this is something to bear in mind (The update in question is KB2999226).

So, when deploying OpenInsight 10 you have the choice of:

  1. Ensuring KB2999226 is installed on the workstation, or
  2. Using the “Visual C++ Redistributable for Visual Studio 2015” setup package instead.

At the moment we are going with option (1), but this post is basically a “heads-up” to make you aware of the choices.

(Disclaimer: This article is based on preliminary information and may be subject to change in the final release version of OpenInsight 10).