Prophet Enterprise (PE)

• Flexible Results: Aggregation results
• Flexible Results: Individual policy results
• Flexible Results: Flexible Results API
• Table and model point optimisations with modules
• Usability enhancements

• Flexible Results: Distributed Database Output
• Results change
• Looped Modules
• Advanced Vector Extensions (AVX)
• Prophet Grid Administrator API
• Improved data transfer protocols
• Size of the batch can be specified for split by model point batch
• Categories of finished jobs
• Improved security
• Streamline machine entry
• Improved error logging

Aggregation results
Aggregation results allow you to generate projection results aggregated by a model point variable of your choosing, called the aggregation variable. Aggregation projection results can be created for up to two aggregation variables per product in addition to existing sub-product aggregation functionality.

Aggregation results are specified in a new workspace object, the Aggregation Results Definition, before adding to the run setting in the Flexible Results section of the Results tab.

The new results definition can also be created, edited and added to your job’s run settings via the API.

Aggregation results output can be viewed directly in the Prophet Studio application, Results Viewer and with Excel Reporting through a new AGG_PROJ_RESULT function.

Individual policy results
Individual policy results were previously only available to clients with a Production Interface licence, they are now also available to users licensed for Flexible Results. Individual policy results allow you to create projection or stochastic output for specified policies in your model point file, up to a 100 policy limit.

The specification of these results adopts a similar approach to aggregation results, via a workspace object which can then be referenced by multiple run settings. Individual policy results can also be set up via the API, read directly in the Prophet Studio application, Results Viewer and with Excel Reporting through a new POLICY_RESULT function.

For further information on either aggregation or individual policy results, see the Prophet Professional 2020 Q2 Bulletin.

Flexible Results API
The Flexible Results API (FR API), which was released in 2020 Q1 for Prophet Enterprise users, has been updated to read both aggregation and individual policy results. The FR API is a .NET Dynamic Linked Library (DLL) that can be used by a bespoke application to directly read Prophet results at scale without the need to first load the data into the Prophet Results Database (PRD).

The Insurance Data Repository (IDR) can now use FR API to efficiently extract, transform and load Prophet results data with minimal changes to any existing jobs that were using PRD as a data source.

Table and model point optimisations with modules
Table and model point optimisations can now be used with products containing modules. Table and model point optimisations can be enabled on the Runtime tab of the job or job template.

For further enhancements made to modules as part of this release, see the Prophet Professional 2020 Q2 Bulletin.

Usability enhancements

• Improvements have been made to speed up the access and display of results in the Prophet Enterprise user interface.
• The performance of opening run settings when they reference base sets in the tables tab has been improved.
• Domain settings have been split between two permissions: Administrative Tasks and Actuarial Administration. The new Actuarial Administration permission grants access to non-system administrator permissions like the ability to amend table locations, job priorities or default upload options. The change does not affect existing roles but does enable custom roles to be more specifically tailored to your environment’s needs.
• Monitor Manager can now update itself when the list of machines in a grid changes without needing to be restarted. It does this by polling the list of machines, and when it detects any changes it adds and removes machines as necessary. The polling rate is configurable but by default is set to 0 so that no polling happens and the original Monitor Manager behaviour is preserved.

• There was an issue with the Push to PE feature so that when submitting multiple run pairs, the second and subsequent job run pair would fail.

  This issue has been resolved, with the following limitations in Push to PE run pairs. For a specific workspace:

  • Run structures must be unique.
  • Run numbers must be unique.

  The preparation stage for each of the run pairs is performed sequentially rather than in parallel. The runtime of multiple run pairs can be performed in parallel.

  You cannot suspend and resume a Push to PE job. Instead, cancel the job and resubmit it once the other jobs for that workspace have finished.

• The Prophet Enterprise functionality that lets a user copy results to an output location, and potentially merge them by run number with existing results in that location, did not correctly work if the location had existing results in it. In such situations, results viewing components such as Excel Reporting could not report on SameAs products in the results: an error, relating to not finding a key piece of metadata required to interpret the results files, was returned instead of the requested results. This issue has now been fixed.
• An inefficiency with the READ_RESULTS function was introduced in Release 2019 Q2 where the Calculation Engine (CE) checked the results frequently to identify whether a requested result was part of a product or accumulation. This involved checking whether the requested product or accumulation was part of the currently executing job, and if not, checking whether it was part of the configured external results location(s), which is relatively expensive in terms of run time. Performing this check repeatedly for every READ_RESULTS call could significantly impact the runtime of products using lots of READ_RESULTS calls, for example Summary library products. If the results were located on a remote location to the worker(s) then the performance impact could be more adverse. This issue affected models where there was a valid external results location configured and a lot of products and accumulations results being accessed were either missing or in the external results location. This issue has now been fixed.
• In previous versions, on the Prophet tab of the ribbon in Excel, if you clicked Options and then cleared the default selection of Show Variable Errors, an “unset nullable” error was returned instead of zero. This issue has now been resolved.
• Prophet 2019 Q2 introduced an issue in which all variables for a particular set of variable-by-time results are loaded into the results cache when a single variable is requested. The issue only affects the CE results-reading runtime functions PROJ_RESULT and STOCH_RESULT and the Excel Reporting worksheet functions of the same name. This can cause a significant memory increase during runs which access a small subset of variables across a large number of these sets, for example a summary product accessing a small set of variables across a large number of products and sub product codes using the affected functions. This issue has now been addressed for results stored using the file system directly but not for results stored using ArangoDB.
• There was an issue with sensitivity documents if the sensitivity value for an assumption was equal to the value defined in the related basis document, and the value in the basis document was defined to be different from the underlying table value. Under such conditions the sensitivity resulted in the underlying table value being used instead of the value defined in the basis document. The issue was present for both cases, when applying sensitivity to the reporting date and to a specific AoC step. This now has been fixed.

Prophet Enterprise 2020 Q2 Service Pack 1

• This update fixes two threading issues in the Prophet 2020 Q2 Calculation Engine, that affect PMPP:
  1. If using PMPP and READ_FLEXIBLE_TABLE it is possible to get a putResults error of the type “concurrent or recursive” calls.
  2. If using PMPP and modules it is possible to get a structured exception when two DLLs both try to load the same table at the same time. Depending on whether fast runtime or fast compile is used in the structure, or whether a module link is set to be looped, a DLL may represent a single module shared amongst module links in the same product or individual module links.

Note: It is recommended that users apply this update if they are using PMPP.

Prophet Enterprise 2020 Q2

• There was an issue with erroneous generation of base sets when referenced directly in a run setting as part of a job. The issue generated a version for the base set, regardless of whether a base set change had been made since the last version was taken. This would then fill the database with many copies of the same base set version, which affected performance. This has been fixed so that now versions are only created when the base set data is unique when submitting a job. This is now consistent with other methods of job submission that use base sets.
• There was a performance issue that caused file system writing for certain types of results files to bypass the Windows file cache, causing some Prophet models to run noticeably slower. This has now been fixed.
• An issue which meant copying run settings from one workspace to another would, on occasion, throw an error has been fixed.
• There was an issue with the Override Results Directory names option which meant multiple run numbers could be stored in the same directory. Validation in the run setting now prevents this from occurring.
• Bugs in the Prophet Enterprise Administration Tool (PEAT), found when changing Data Storage from By Compartment to By Domain and vice versa, have now been fixed.
• There was a bug in the Prophet Grid Administrator which meant a spurious error message would appear if your web server address included a port number. This has now been fixed.
• The Prophet Enterprise runlog has been made more resilient to instances where the ArangoDB server goes down in the middle of job preparation. In this scenario, accurate start and finish times for preparation are now shown.

Prophet Results Database

• In Prophet Enterprise Release 2019 Q2, the Create a results file for download option would create a zip at a different level in the results file structure compared to previous releases. This meant that Prophet Results Database uploads using the -cri argument would not succeed. This has been fixed so that the results are once again zipped at the pre-2019 Q2 level.

For some models, the speed of reading and writing results is the crucial factor affecting execution times. The windows file system technology traditionally used by Prophet can form a bottleneck for these models. As data volumes increase (because of regulatory pressures such as IFRS 17), the number of models affected by this also increases, and the execution time becomes an important issue. The distributed database technology at the heart of the Flexible Results module supports optimised disk usage, parallel writing and file compression, allowing users to address this bottleneck, and giving them greater control over the results data produced by the models.

Larger models benefit from the scalability achieved through the faster writing and better organisation of the results output, without relying on expensive specialised hardware.

In the 2019 Q2 release, users licensed for Flexible Results can configure Prophet Enterprise to output results to ArangoDB. ArangoDB is a NoSQL database system supporting parallel writing and file compression, and can be installed from the Prophet Enterprise installation disc. For this release ArangoDB must be installed on a Linux operating system. This is due to the greater
performance achieved on Linux. More information on ArangoDB can be found in the Prophet Enterprise Technical Guide, and on their website at https://www.arangodb.com/.

A new tab has been added to the Prophet Enterprise Administration Tool to define Arango configurations. These configurations
detail which database and server the configuration applies to, and how workers connect to the specified database when jobs are
submitted.

The data storage setup tab has also been updated to allow users to choose a results storage mechanism. Users licensed for Flexible Results can either choose to output results to the Prophet Enterprise file system as normal, or to select one of the previously defined Arango configurations. This configuration is applied at a domain level, so different domains can be set up to output to different Arango databases. This will apply to all future jobs submitted after the configuration is set, providing users the flexibility to switch between storage locations over time.

Text results and runlog information will continue to be stored under the job folder on the Prophet Enterprise file system, even if the other results are stored in Arango.

Results change
To improve the scalability of Prophet under our Flexible Results project, enhancements have been made to how results are stored in Prophet Enterprise. Flexible Results is a new licensable feature for Prophet Enterprise users, however these changes will be seen by all users including those who are not licensed for the Flexible Results feature.

To optimise these changes the names of the files stored in the results folder have been altered. It should be noted, however, that this has no effect on how results are presented in the Prophet Enterprise user interface, and the same names for the results files used previously are retained here for your convenience. Results produced by Prophet Enterprise can still be used with other Prophet applications and you can still view results produced by previous versions of Prophet in this latest release.

Looped Modules
Prophet Enterprise 2019 Q2 supports the looped modules functionality introduced in Prophet Professional 2019 Q2. Looped modules workspaces created in Prophet Professional can be uploaded to Prophet Enterprise. For more information on Looped Modules please see the Prophet Professional 2019 Q2 Bulletin.

If you are interested in seeing how Modules and Looped Modules can be used please speak to your Sales Representative or refer to the user manuals.

Advanced Vector Extensions (AVX)
Looped Modules works best in conjunction with Advanced Vector Extensions, which allows the loops of the Looped Module to be run in parallel. AVX is a set of CPU instructions that enable vectors of integers, singles or doubles to be processed simultaneously. This allows the processor to calculate a lower number instructions on the same amount of data and thus increase
performance. This will be of particular benefit to cloud users, for whom runtime directly affects costs.

Each worker process is now capable of executing different Single Instruction Multiple Data (SIMD) instructions concurrently. For example, the following four instructions might be sent to the CPU:
18 + 11
12 + 5
6 + 8
32 + 2

Traditionally, this is processed by the CPU as four separate instructions which must be undertaken one at a time (ignoring complexities around the difference between Floating Point Unit (FPU) and Arithmetic Logical Unit (ALU) processing and how these can be parallelised). With AVX, this set of instructions could be processed simultaneously, by refactoring this set of
instructions as one vector addition:

Performance is fundamentally driven by how many instructions the CPU can perform per second. By reducing the number of instructions required to process the same data, AVX can significantly increase performance.

AVX is licensable functionality, and you must also have a Prophet Enterprise licence. Your model must also make use of the Looped Modules functionality as it is the loops that are optimised to use the AVX functionality.

If you are interested in finding out more about AVX and how it can help then please speak to your Sales Representative.

Prophet Grid Administrator API
We have developed a new API for use with Prophet Grid Administrator. This API enables the use of the key functionality from Prophet Grid Administrator via an API, for example:

• Adding/deleting machines from a grid
• Adding/deleting machines from domains
• Adding/deleting/editing machine groups
• Adding/removing machines from machine groups
• Adding/removing machine groups from compartments

This API is accessible via PowerShell or .NET, and so is able to be used by Prophet Control Centre, or whichever mechanism you currently use to access our existing APIs.

Improved data transfer protocols
We are always seeking to improve the security, reliability, and speed of our software. To that end, for 2019 Q2 we have deprecated BITS and moved to WCF chunking as our data transfer mechanism.

• Upload and download of data from Prophet Enterprise servers are now handled by WCF.
• References to BITS in the Studio UI and configuration files have been removed.

Size of the batch can be specified for split by model point batch
The Model Point Batches split type has been improved. In the run setting you can now specify the number of model points that each batch contains rather than having to use the default of 10,000. This provides flexibility to tune your model performance.

Categories of finished jobs
Finished jobs were previously read-only entities. This could cause administrative issues when new categories were created in the Prophet Enterprise environment. From Release 2019 Q2 you are able to change the category of finished jobs. The rest of the entity remains read-only to protect the audit trail.

Improved security
You are now able to configure the SMTP mail server to send emails using SSL for notifications.

We have additionally made other security enhancements, which are invisible to end users.

Streamline machine entry
In previous releases you could use the Monitor Manager config file as a mechanism to add machines to your Prophet Enterprise grid. This approach is no longer supported and has been changed to simplify machine administration.

Improved error logging
Some error logging has been improved for the API, particularly with regard to serialisation errors.

Note: From this release we no longer support 32-bit compilation of Prophet code and calculations need to be performed using the 64-bit runtime architecture. Infrastructure that only supports 32-bit will not be able to be used with Prophet Release 2019 Q2.

• A bug which sometimes prevented you from accessing the runlog via the web service when using HPC has been fixed
• A bug which caused time-outs when filtering a very large list of finished jobs that were using many parameters has been fixed
• A bug which caused the scroll bar to not appear on the categories and parameters window of the job creation wizard has been fixed
• An issue which meant there could be duplicate email addresses in the Notification tab of the Job Submission dialog has been fixed
• A user interface issue with nested structures jobs which meant the same structure was duplicated in the versions dialog box has been fixed
• A bug which sometimes caused the runlog to not be indexed after resubmitting a failed job has been fixed
• A bug with the Allow Concurrent Model Point Batches check box in a product which meant that the correct value could be lost on creating a new version of a workspace has been fixed
• Having an expired password on the target environment when performing a promote caused an error. This has been fixed and users are now correctly prompted to reset their password.
• An issue with reconnection via the API after a session is lost has been fixed
• Some issues with errors produced during a database upgrade have been fixed