Release |
Enhancement |
Addressed Issue |
2024 Q4 |
Prophet Enterprise
General Usability Improvements
Job Worker Metrics
- A new option has been introduced in Prophet Enterprise to collect machine information from workers during a job, this feature can be used to help troubleshoot problems during a job or gauge the optimal level of resources a job requires for more cost-efficient worker machine specification.
- Collection of metrics is turned on in the Logging section of a Job or a Job Template.
- The logs are collected during the run and at the end of the run they are archived and stored in a subfolder of the job results as two archives: one for job preparation, and one for job execution. This is particularly useful when workers are temporary cloud machines such as PMCS – MMS environments where an arbitrary number of workers can be spun up and then automatically shut down at the end of the job.
- A variety of metrics are supported covering the categories of processor, memory, disk and network. All metrics are machine level to capture a complete picture of what is happening to the worker.
- Some metrics are a single static value captured at the start of the job, such as disk sizes and total physical memory. Other metrics are time-dependent and sampled approximately once every second over the duration of the job. Both types of metrics are stored in separate files in the archives with a row per time sample for the time-dependent data.
- The metrics can be deleted, without deleting the entire finished job, from within Prophet Studio via the finished job object. If a finished job is deleted, then its metrics will be automatically deleted.
- The metrics can be downloaded from Prophet Studio via the finished job object.
Flexible Results
Individual Policy Projection for All Policies
- The Prophet 2023 Q2 Service Pack 4 release enhanced the Flexible Results feature Individual Policy Projection Results functionality such that these results could be produced for all policies. Previously this was restricted to a maximum of 100 policies.
- Due to the potential to output a significant amount of results data with this feature, users must be using Prophet Enterprise to run the model and be outputting this to an Arango Database.
- The documentation for this enhancement has now been fully incorporated into the Help and Prophet manuals.
- The limit imposed on the Individual Policy Projection for All Policies output has been made more flexible in 2024 Q4. Previously there was a limit of 20 variables and 10 years that could be outputted. This has been updated such that there is now a limit of 3600 values that can be outputted, where this is calculated as ‘The Number of Variables’ multiplied by the ‘The Number of Time Periods (in months)’. This means that you are now able to run with a much larger number of variables or time periods as long as you adjust the other accordingly. Note that each element in an array variable counts as an individual variable with respect to this limiter.
A Note on Arango
- ArangoDB clients are advised to make sure that they keep up to date on the version of ArangoDB that they are using, Arango only support a particular version for a maximum of 18 months after its release and this means that you may have to upgrade your Arango databases more frequently than your version of Prophet. If you are using an out-of-date Arango version, please contact us for assistance upgrading.
- Additionally, we advise all Arango users to deploy Promotheus or Grafana monitoring tools. Should you ever need to contact ArangoDB for support, the logs available from these monitoring tools will help to provide more information on any issues that you may be experiencing which should reduce the time spent troubleshooting.
- In PEAT (The Prophet Enterprise Admin Tool) when creating a collection, you are presented with options to set the number of shards and the Replication Factor. Documentation is being added to the PEAT Help so that administrators can be informed as to what to set these values to. Please note that these settings cannot be altered after the collection has been created. It is therefore important to consider how big your cluster may become and to set the number of shards to a value that is big enough to cover that growth. If you use a cluster where the number of shards is less than the number of database servers then one or more of those servers will effectively not be used by Arango to store the Prophet results.
Prophet Enterprise API
- We have introduced two new methods in Prophet Enterprise API:
- PEFinishedJob.MoveResults(session.GetResultsRepository(name))
- PEWorkspace.MoveResults(session.GetResultsRepository(name))
These methods will enable users to move finished job results from one location to another. This functionality is limited to Super Administrator and Business Administrator roles.
- Users are now able to set the Outer and Inner split type of a Nested Structure job using the Prophet Enterprise API.
- We have expanded the GetRole method capability in the Prophet Enterprise API to allow users to get the permissions associated with the requested role and information such as: “Inherits From”, “Process Access” and “Object Access”. The GetRole method was previously only returning the name of the role and a GUID only.
- We have introduced the GetUsersForAdLogin method to allow system administrators to get the list of all the Prophet Enterprise users linked to their Windows (Active Directory) account.
- We have updated the Prophet Enterprise API for Aggregation Results Definitions. It is now possible to list all selected accumulations in the Aggregation Results Definition.
- We have introduced a new method GetJobs which will give a user the ability to submit jobs in batches.
PE Grid Administrator API
- The PE Grid Administrator API can be now used to add or remove the output paths for the Results Storage Definitions specified in the PE Administration Tool.
|
Prophet Enterprise
- There was an issue related to the usage of parameters in different objects of the Enterprise system. In cases where a user incorrectly used a parameter, for example, by not typing an opening or closing square bracket, the object would get corrupted and could not be opened after saving. The issue is now fixed.
- We’ve fixed an issue where the runlog was failing to navigate to the errors section of the runlog when clicking from the links available on the summary page.
- There was an issue where a user was not able to open a Finished Job and was getting a missing entity error message. The issue was related to the system licensing component and is now fixed.
- We have corrected several issues related to the display size of certain dialogs within the Prophet Studio user interface. The bugs could have been encountered when working with different parts of the system, including login and server selection dialogs, job screens, run structure and run setting screens, job creation or job submission wizard dialogs, export workspace dialog, filters selection for queries, categories and parameters edit screens. The root causes of the issues were the incorrect handling of display scaling setting other than 100%, or the lack of full support for operating system languages other than English. These issues are now fixed.
- There was an issue where lists of objects were not refreshing even if elated refresh rates were specified in Domain settings. This was affecting Finished Jobs, Active Jobs and Active Machines. The issue is now fixed.
- Some users previously encountered an issue where the Aggregation Results Definitions and Individual Policy Results Definitions were updated on a workspace update even when these objects were not selected to be updated. We have fixed this issue.
- There was an issue related to workspace upload and storage when the environment was setup to store data by compartment. Users could encounter some folders being deleted when they shouldn’t be. We have fixed this issue.
- Archiving, or using the Results Move functionality, to move results to a location that already contains results in Prophet Enterprise will now correctly fail.
Prophet Enterprise API
- We fixed a bug relating to slowness in deleting finished jobs in Prophet Enterprise when doing this using the Prophet Enterprise API.
- We addressed an inconsistency between Prophet Studio and Prophet Enterprise API when creating jobs with the same name. Users are now able to create jobs with the same name using the Prophet Enterprise API.
Calculation Engine
- Cancelling a Prophet Enterprise Nested Structure job may sometimes return that the job had failed rather than cancelled. This has been fixed.
- The performance of deleting jobs when using ArangoDB to store Prophet results has been improved significantly.
- We advise that Prophet data file servers are not compressed. File compression is more likely to cause Prophet run issues. A warning is now given in the runlog if your file server is compressed.
- An issue was found when performing a job with many worker processes (1000+). In some cases, not all the worker processes would connect correctly to the task master machine organising the job and would result in a connection actively refused error. The underlying causes were environmental, but the worker system has been enhanced to be more robust – each affected worker process will now retry a couple of times with a small delay between attempts to give the task master machine more time to initialise and accept the requests or prevent heuristic firewalls from mistaking the large number of connections over a small time period as a denial of service attack.
|
2023 Q2 |
Prophet Enterprise
- We have continued to improve the performance of the Prophet Enterprise Studio application and most of the operations available to Prophet Enterprise and Assumptions Manager users, as a result, Studio should appear to be more responsive to user actions performed in the interface.
There are three main areas of improvements:
- Displaying Job entities (Job Templates, Jobs, Active Jobs) is approximately 5 times quicker and potentially faster for Job Templates.
- Creating Job Templates and Jobs from Job Templates is approximately 2 times quicker.
- Modifying entities with options like edit, rename, save edit mode is approximately up to 1.5 times quicker. The entities benefiting most of the improvement are jobs, run settings, run structures, and job templates.
Note: Note that all performance figures provided are model and environment dependent therefore users may have a different experience in comparison to the FIS test environment results quoted above.
- We’ve given the Prophet Enterprise Studio application user interface a new, modern look. Dark blue contrasting with white and light grey makes the highlights and selections clearer. Newer looking icons modernize the look of options and features on the ribbons. Users should find the application interface to be more consistent with the Prophet Professional application.

The refreshed interface also includes some usability improvements. There is now the ability to drag and drop tabs to reorder them. Users can now drag a tab with a left mouse click and move it to any other place in the tab list. Users who tend to work with many open entity panels should find this useful.

- A Prophet Enterprise user can now filter the list of Finished Jobs with new query functionality. Rather than listing all the finished jobs, it is now possible to set-up and run a query and limit the list according to the specified filters.

This feature intends to improve the performance of the Finished Jobs listing by allowing for a more targeted query definition. Moreover, a user can open multiple Finished Jobs Query panels at a time, each with different combination of filters.
From the Finished Jobs Query, users can select multiple jobs using a checkbox selection column and then apply an action a desired action to all the selected items simultaneously. The list of actions covers: Open, Re-Submit, Download Results, Move Results, Delete Results, Delete System Logs or Delete.

- In this release we have replaced the XPS format with PDF. PDF is a more flexible, modern, and more performant format. This change affects:
- Producing runlogs post run, these are now output as PDFs
- Saving Assumption Manager entities comparison
- Audit Log exporting
- Printing Runlogs from Results Viewer
- The new PDF runlogs now include an interactive content page to help navigate through the document.

We’ve also supplemented the content page with bookmarks. This makes runlog navigation even more efficient.

- In the previous release, we introduced an option to Merge on conflict when promoting and restoring Basis Documents. In this release, users can additionally apply the Merge option for Parameters and Categories.
- With this release have made more Prophet Professional objects visible to Prophet Enterprise users. The intention is to help users ensure the job setup is correct before submitting a job. Previously users had to export the workspace, open it in Prophet Professional to validate the setup of certain objects. The following objects are now visible in Prophet Enterprise Studio, making the validation of these objects much more efficient.
- Variable Groups – A user can now check the members of each Variable Group by opening the Variable Groups tab on the Workspace panel. This panel is read only.

- Accumulation Details – A user can now check the members of each Accumulation by clicking its name on the Accumulations tab of a Run Structure. This activates a new Accumulation Details tab. This panel is read only.

- Structure Link Details – A user can now check all of the details of a Structure Link by clicking its name on the Structure Links tab of a Run Structure. This activates a new Structure Link Details tab. This panel is read only.

- Prophet Enterprise users are now allowed to specify how inner run pairs (structure links) are split. Previously inner run pairs were limited to: No Split or Split in the same way as the outer. In this release, each inner run pair can now be split in a different way to the outer run pair, or additionally, a different split method can be selected for each individual structure link . There are three different options available when setting up the split method. Each option allows a choice of Split by Product, Simulation, or None.
- To select the same split method for all Structure Links, the user should select Product, Simulation, or None for the Inner Split Type field in the Job definition.

- To specify the split method in the inner run setting, the user should first select ‘Specified in Inner Run Setting’ for the Inner Split Type field in the Job definition. Then edit the Split Setting details on the Run Parameters tab of an Inner Run Setting:

- To specify the split method for each structure link, the user should first select ‘Specified at Structure Link Level’ for the Inner Split Type field in the Job definition. Then edit the new Structure Links tab of a Run Setting, for each structure link, you can override the split type defined in the Run Parameters tab.

- A new option on finished jobs has been introduced to allow results to be moved to another location. By doing this, results can be moved to cheaper alternatives or can be moved from servers that may be nearly full. This action is limited users that have licensed the Flexible Results module. It is also limited to Super Administrator or Business Administrator roles. Please note that only the results are moved; the other job files are maintained in the original job location. This move also maintains the links in Prophet Enterprise such that the job can still be viewed; results accessed and runlog opened. Excel Reporting and Prophet Queries set up to access the original job location will also work seamlessly and paths do not need to be updated to the new location.
In order to make use of this functionality the available paths are set in the Prophet Enterprise Administration Tool (PEAT).

- We’ve improved the user feedback for Active Jobs by including an additional option to display information on the current warnings and errors count.
To have this information displayed on the Active Jobs list, the Errors or Warnings columns must be selected in the Fields Chooser. The information is updated according to the Active Jobs Refresh Rate Options.
Prophet Enterprise API
- Users will now be able to trace a finished job using the GUID generated in an active job in Prophet Enterprise API.
- We have updated the Prophet Enterprise API to enable users to get the status on whether the following Advanced Settings section in the Run Structure are enabled or disabled. Users can also set the status to enabled or disabled on these settings using the Prophet Enterprise API.
- Enable fast compile
- Disable module code safety checks
- We have updated the Prophet Enterprise API to enable users to get the status on whether the following settings in the Run Setting are enabled or disabled. Users can also set the status to enabled or disabled these settings using the Prophet Enterprise API.
- Binary Individual Model Point Results Format in the Individual Model Point Files section in the Run Setting
- Run Selected Policies Only in the Projection section in Run Setting.
- Debug Dynamic Results in the Additional Files section of the Run Setting.
- Users will now be able to retrieve a list of all the compartments from the domain they are currently logged into using Prophet Enterprise API.
- We have introduced a new method to fetch the username for CreatedBy and LastModifiedBy in Prophet Enterprise API. Users previously using “var user = workspace.Data.LastModifiedBy.Name” will now need to update their script to “var user = workspace.Data.LastModifiedBy.GetName()”.
PEGAdmin API
- We have updated the PEGAdmin API to enable the ability for users to add or remove the output paths in the Results File Paths section of the PE Administration Tool.
- Users are now able to use the PEGAdmin API to enable or disable the “Allow Results Storage Override” settings in the Data Storage Configuration section of the PE Administration Tool. User is also able to get the output path selected in the Data Storage Configuration section of the PE Administration Tool.
|
Prophet Enterprise
- We have continued to improve the READ_RESULTS functionality when using External_Results functionality in Prophet Enterprise, making the behaviour more consistent across the different options. External_Results allows you to read results between jobs. It was previously possible to read across run numbers using accumulations or READ_RESULTS. However, the order and definition of the External_Results would affect which run number was accessed and therefore
inconsistent results could be produced. We have now ensured that you can only read results from a job of the same run number ensuring that results are consistent when more than one external result is accessed.
- It is however still possible to list the same run number more than once as an external result. We would advise caution here as inconsistencies can occur. To help the user, we have made some changes in this area. If two (or more) jobs have the same run number, then results will be read in the order that the jobs are listed. Once a job is found to contain those results for the product or accumulation required it will not continue to read the next job. Previously it would try to access all jobs in the list even if the product or accumulation appeared in more than one. We have also added a warning so that users are aware that they should be careful when accessing more than one job with the same run number.
- We have also introduced warnings if you try to accumulate results from the same run number, where the spcodes are different or where the array dimensions are inconsistent. Models of this nature are unusual. However, if you have a model that may make use of the ability to accumulate across jobs then we would advise reviewing any new warnings and checking that you are happy with the results.
- Previously when reading an Analysis of Change (“AoC”) run in Prophet Enterprise which has more than one run number from a job that only has one, then that job would fail. This has been fixed.
- There was previously an issue with the worker machines getting stuck in a “starting” state when submitting large jobs. This was due to the job status message exceeding a limit on the maximum message size that the system could send. This has now been fixed by increasing the limit on the message size.
- Previously there was an issue with Results Diagram View for module links where they did not display correctly. This has now been fixed.
- A user was not able to submit a job after uploading tables only for a workspace. This has now been fixed.
- An issue was found with Arango when the number of jobs\collections increased to around 1000 in the database. This caused a gradual slow down with preparation until the Prophet Enterprise system became unresponsive. We have fixed this by using a single collection within the database for all Prophet Enterprise jobs. Performance is now more consistent as the number of jobs increase.
- There was an issue with results access setting in the web.config file. Results and Runlog access options in PE Studio were ignored when set to the default. It has now been fixed.
- Users trying to create a new domain by using a PE Administration Tool could encounter an error message preventing them from doing this. The issue is now fixed.
- Jobs were not submitting for a workspace which was updated with selected Objects in Workspace option and only Tables in Workspace set to All with all other objects to None. This has now been fixed.
- The issue whereby the submitted by information (and some other IDs) were missing in the Job History Report has now been fixed.
- There was an issue with selecting the filter dimensions in PE Results view. In some cases, the list of products or accumulations on the filter list were incorrect or even empty. It has now been fixed.
- The date and time format for the dates used in the parameters and written to the runlog was used from the web server settings and not from the client machine settings. This has now been changed.
- The option to “show only parameters that are in use” on the job creation wizard was not showing some parameters despite them being used in the job. User could still see all parameters by unselecting the “show only parameters that are in use” check box as a workaround. The issue has now been fixed.
- When submitting jobs with parameters on an environment with multiple domains some of the finished jobs properties, like Submitted By, Machine Group or Compartment, could be missing. The issue is now fixed, all the properties should be correctly displayed to the user.
- When a user tried to modify the “Future Accumulation Period” option for an Inner Run Setting and close the entity, they would get a message asking to save the changes even if the changes have already been saved. The issue is now fixed, and misleading messages updated.
Prophet Enterprise API
- There was an issue where a user could add a product to the structure which already had an existing product with the same name. This has now been corrected and is no longer possible.
- Using the Prophet Enterprise API to get the ResultsLink, PDFRunLoglink, CompartmentLink and RunlogLink properties of a finishedjob.data now returns the correct folder path location. This was previously returning blank.
- We fixed an issue in Prophet Enterprise API relating to the Get-ProphetSession in Powershell not returning the PE session object. As a result, none of the methods can be called, this has now been rectified.
- We fixed an issue in Prophet Enterprise API relating to user adding table locations to a compartment is not possible and will return an error message.
- There was an issue where a Prophet Enterprise API user uploading a workspace, the system will create a temp file which contains all the workspace files and this is not deleted at the end of the operation. This is now fixed.
|
2022 Q1 |
Prophet Enterprise
Prophet Enterprise API
- We have expanded the scope of deleting versions of Prophet Enterprise workspaces. Users can now delete minor versions of Prophet Enterprise workspace objects in this release. This should help in maintenance and cleaning of large environments. Previously, users were only able to delete major versions of Prophet Enterprise workspaces.
- In Prophet Enterprise and Assumptions Manager APIs, when executing the Promote method the value of Execute on Server is by default set to True.
- We have expanded the API in Prophet Enterprise job submission to allow users to overwrite machine groups and defer the start time for a job.
- We have added a new method to create compartments using Prophet Enterprise API. With the right permission, users can create more compartments and have a more organised environment.
- For any jobs submitted in Prophet Enterprise API, users can choose to output job-specific runtime system logs.
- Users can use the Prophet Enterprise API to select data storage configuration on a per job basis in jobs or job templates. It is now possible to choose between different ArangoDB configuration vaults or different file systems to store the job results.
- We have now provided a user guide for the Prophet Enterprise Grid Administrator (PEG Admin) API. This also includes some examples for PEG Admin API commands in C# that can be used as a reference for script developments.
- Users can use the Prophet Grid Administrator (PEG Admin) API to set which compiler to use in the Prophet Enterprise domain.
- In this release, we have added a method to allow users with the right permissions to set server fields in Prophet Enterprise Grid Administrator.
- Users can parameterise the workspace name and job name in the Job Dependencies tab of a job template using Prophet Enterprise API.
|
Prophet Enterprise 2022 Q1
- When the ArangoDB connection is lost, for example if the service is stopped, the job execution status now shows Failed immediately, instead of hanging indefinitely in a Running status. The original issue has now been fixed.
- In earlier versions, if an ArangoDB job did not output correctly due to environment issues, for example an overloaded cluster, the job would sometimes not be deleted. This issue has now been fixed.
- Suspending a Nested Structures job whilst using ArangoDB as results storage would result in the job restarting. This has been fixed.
- A Prophet Enterprise issue in which split-by-inner-simulation nested structure runs released worker machines too soon has been fixed. The issue manifested when the outer run pair had significantly fewer simulations than the inner run pairs and multiple jobs were being run. Split-by-inner-sim nested structure jobs now hold onto worker machines that are initially assigned to them.
- Stochastic and stochastic summary results were previously taking a long time to open in Prophet Enterprise Studio. This time has now been reduced.
- Prophet Enterprise grids did not correctly detect the number of cores in a worker machine when determining how many worker processes to initiate. This regression has now been fixed and only affected workers with more than one Windows Processor Group; by default, this would only apply to machines with more than 64 cores. The original issue has been fixed.
- We’ve removed an unnecessary limit of 100 rows when copying data from a runlog message grid in Prophet Studio to the clipboard. The number of rows is no longer an issue.
- The Stochastic Summary Results section of the runlog was missing. This has been fixed.
- The AVX options were not copied correctly if the job template was duplicated with a PE API call. This issue is now fixed and did not affect Prophet Enterprise Studio.
- The Validation button on the Open Connection dialog box previously did not return a message when successfully connecting to the server. This has now been fixed.
- There was an issue with the cancelation of certain long-running tasks performed by the server, for example when importing a large workspace. This has now been fixed.
- There was an issue uploading workspaces where a previous version existed, and the new version was missing some of the previous accumulations. In such cases, run structures which were referencing the removed accumulations in the original version were not accessible to edit or read only. In this release the problem has been addressed.
- We’ve fixed some issues with Database Upgrader for environments with specific regional settings.
- Incorrect warnings about unknown machines running jobs were written to Scheduler logs on some environments. These have now been addressed.
- The results reader logic was not correctly reading the results metadata when one or more jobs used the Prophet Enterprise Copy Results functionality and the same destination location. This issue affected models using modules and results reading functionality such as Excel Reporting. This has been fixed.
- System logs were not visible in PE Grid Administration Tool (PEAT). This has been fixed.
- There was a prior suspend and resume issue for split-by-simulation jobs. Previously, depending on exactly when suspend and resume happened, the runlog could state the run execution had failed when in fact all simulations had completed successfully. This issue has been addessed.
- Parallel model point processing models with table errors could sometimes crash due to an introduced thread safety issue. This has now been fixed.
- We have made improvements to the thread safety of reading results (READ_RESULTS) inside models running under parallel model point processing. Without these improvements you could randomly get Read results failed – device or resource busy errors depending on the extent to which results are read into the model (the more results reading heavy the model, the greater the chance). These issues have been addressed.
- In certain circumstances, models using table and model point optimisations and same-as products would not return the correct product name using PROD_NAME. Instead, it would return the name of the product the same as product was based on. This has now been fixed.
|
2021 Q2 |
Performance enhancements
In Release 2021 Q2 Service Pack 9 we have made some targeted improvements to most of the core user interface and API operations which are regularly performed by Prophet users on a day-to-day basis. Those include:
- Improvements to speed up workspace import from a PBUZ file, for workspaces with many products and accumulations. On average users should notice 40% shorter times to import, and even 90% shorter for workspaces with a massive number of products and accumulations.
- A redesign of the way the user interface displays products and accumulations together with the workspace. This leads to a much quicker response time to display of lists of large workspaces.
- A user interface performance improvement for Prophet Enterprise systems with large numbers of parameter definitions. As a result, a user may observe quicker times to display lists of finished jobs and active jobs. We have observed improvements of up to 90% for finished jobs and 50% for active jobs.
- Improved interactions with job templates and jobs. Now times to open, edit or save these entities are at least 75% shorter. The time needed for job submission has also decreased by 15% on average and even more for submission of multiple jobs at the same time.
All of the improvements impact the responsiveness of Prophet Enterprise API calls like: Create Job Template, Edit Job Template, Create Job, Get Job, Edit Job, Submit Job but also Get All Workspaces, Get Active Jobs and Get Finished Jobs.
User interface enhancements
In Release 2021 Q2 Service Pack 9 we have made improvements across several parts of the user interface:
Run structures and run settings
With Release 2021 Q2 Service Pack 9 a Prophet Enterprise user with appropriate permissions will now be able to create a new run structure or run setting and copy these two entities between different workspaces. New options are available in the user interface and via the Prophet Enterprise API (using either C# or PowerShell).
These changes could save time and effort when importing new workspaces; promoting workspaces as well as performing Prophet Enterprise workspace updates. This improvement also enables users to write more powerful Prophet Enterprise API code for actuarial model changes.
Backup and restore
We’ve improved the robustness and stability of backup, restore and promote operations. The enhancements include: n Introduction of a temporary backup file to prevent the file being copied too early.
- A series of improvements for backing up and restoring .amb files with large entities.
- Displaying a progress bar when the system validates dependencies for selected entities.
Role permissions
Unlocking entities in Prophet Enterprise Studio has historically been restricted to users with the Super Administrator role only. For this release, there is a new permission which allows the ability to unlock entities to be granted to other users who are not Super Administrators.
Prophet Enterprise makes use of entity locks to ensure that conflicts are not created when multiple users try to change the same thing. The locks are usually automatically released when the operation completes. It is possible, however, for the lock to not be released properly and in this circumstance, historically, only a Super Administrator or the user themself would have been able to unlock the object.
The new permission is a process access permission type, which can be granted to any user with a default or custom role. The unlock operation can be used on any locked entities and works irrespective of who originally locked the entity. Therefore, we recommend granting this permission to any admin user who may need to control user access and entity creation in the environment.
Prophet Enterprise API
- With Release 2021 Q2 Service Pack 9 two new commands: DeleteVersion and DeleteChildVersion have been added to the Prophet Enterprise API. These can be useful for cleaning up a Prophet Enterprise environment where many versions and sub-versions have been created over time and are no longer required.
- Prophet Enterprise API users can now perform a backup, restore, and promote with the Execute on server option enabled. The new parameter is optional and is turned off by default.
- Prophet Enterprise API users are now able to upload all files under a workspace in the same way as the existing user interface functionality.
- There is a new method which allows Prophet Enterprise API users to delete a workspace version.
- The Prophet Enterprise API documentation has been updated to show all available methods.
PEGAdmin API
- Users with the PEGAdmin API licence can now check the status of the Prophet Enterprise Monitor Manager service via the API. API commands have also been included to start, stop and restart Monitor Manager.
Calculation Engine
- We now support Prophet code compilation using the 2019 edition of the Microsoft Visual 2019 C++ compiler. We no longer support compilation using the 2015 edition.
Plugins
- It is now possible for Prophet Enterprise parameters to be used in plugin related (Intex and Andrew Davison specifically) run settings. This new enhancement allows a user with Submit job permissions – but not with either Edit job template or Edit job permissions – to set the value on job submission. This enables users to change the plugin value or locations at submission without altering the job template.
Documentation enhancements
- The Prophet Enterprise Technical Guide has undergone a thorough review to bring it into line with latest guidelines for Prophet Enterprise installation. The guide is now more comprehensive with the steps required for setting up a fully functional environment laid out chronologically rather than grouped based on Prophet Enterprise component.
Licensing
The following improvements have been made to the Prophet licensing technology:
- In the Client Licence Server (CLS) there is new Export functionality, allowing you to export the filtered on-screen data to a csv file. You can export data from the following tabs: Licences, Licence Consumptions, Synchronisations, Users and the Clients details tab (for a specific client machine).
- CLS users now have roles. User permissions have been split into two roles – Administrator and Licence Administrator – allowing you to better manage CLS user access and entitlements.
- The Administrator role is for those users responsible for server maintenance, ensuring its connectivity with both the FIS Licence Server and Prophet users. These users also have permission to create and delete CLS users.
- The Licence Administrator role is intended for those users tasked with administering the licences themselves, for example, adding users to a group licence and looking at licence consumption history.
- In the CLS portal, users now have the option to remove multiple group licence users in one go from the group licence Users tab. Previously, users would have to be removed separately.
- A new option has been added to the CLS installation giving you the ability to change the CLS hostname to the fully qualified domain name. Previously the hostname was set to the ‘simple’ name of the server on which the CLS was being installed by default.
- You can now view a detached licence’s products and features on the detached tab in Prophet Licensing.
- The CLS user documentation is now provided as HTML5 help embedded in the CLS portal, to make the documentation much easier to access. The help is completely accessible from within the web browser, with no read access to the CLS installation directory required.
|
Prophet Enterprise 2021 Q2 Service Pack 9
- Projection text results (PRN files) were written twice; by the workers after each product and again by the taskmaster at the end of the run pair/job. This could lead to longer run execution times. They are now only generated by the workers.
Prophet Enterprise 2021 Q2 Service Pack 7
- An issue of being unable to view any variables in Module (Results) Diagram View for pre-decision module links has been resolved.
- There was an issue with PMPP mode running significantly slower than non-PMPP when the model was making intensive use of READ_RESULTS function due to a critical section in the error handling logic. This has now been optimised and will also benefit models that make intensive use of table reading functions and plugin calls where similar logic applied.
- The default logging settings for errors and warnings were found to be not persisted from Prophet Professional to Prophet Enterprise. This has been fixed.
- An issue of base set status not correctly showing in the base set document panel when the user updated documents and templates linked to a base set has been fixed.
- When the user enters Edit mode on an Assumptions Manager job definition, the sensitivity document that is selected will now remain selected.
- An issue of reporting date parameter selection throwing an exception on basis document submission has now been resolved.
- Workspace list is now sorted in the dropdown box of the Assumptions Manager Job Submission dialog box.
- A reduction in frequency of validating AD user per web request has been implemented.
- There was an issue where if you tried to open a very large finished jobs list, Prophet Studio could become unstable or time out. This issue has been resolved, and accessing large finished jobs lists in PE API has also been improved.
Prophet Enterprise 2021 Q2 Service Pack 5
- System logs were not created. We have fixed that so they now can be seen in PEGAdmin tool.
- Users whose client and server machines used different time zones would see a delay in the yellow banner being displayed in the basis document after updating an underlying base set. This is now fixed.
- The Assumptions Manager report, which is produced with an Assumptions Manager job or when a user generates PP data, was showing incorrect reporting dates for unlinked tables being used in the job. The issue has been fixed.
- We have added information on the version of tables to the Assumptions Manager report. It is now consistent with the information in the basis document version used for the Assumptions Manager job. The change affects the Assumptions Manager report produced with an Assumptions Manager job and when the user generates PP data.
Prophet Enterprise 2021 Q2 Service Pack 3
- There was an issue in which read results in a run failed. This was caused when devices or resources were busy, and resulted in multiple errors in the run log as well as the run failing. This issue has been addressed, and the run completes successfully.
- There was an issue whereby Excel Reporting could not access results when using copy results functionality and when the model contained modules. The error reported an issue with a missing metadata file. This has now been fixed allowing such files to be read correctly using Excel Reporting. Note this issue has also been fixed in 2020 Q2 Service Pack 4.
- There was a problem where Prophet Enterprise would report and use an incorrect number of cores on a worker machine. This has been fixed.
Prophet Enterprise 2021 Q2 Service Pack 2
- There was an issue installing the Client License Server because some machines didn’t allow the user to elevate PowerShell execution privileges. This has been fixed in the CLS installer.
- It is now much quicker to open and edit basis documents. The improvement is also noticeable for basis documents updating with yellow banner to apply any base sets’ changes.
- Run logs for results stored to Arango Database will now open correctly. The change was made to the Open Run Log option to handle both the job path and the results path typed into the Select results directory dialog box.
- There was an issue for workspace update in cases where the user deleted some or all of the accumulations in Prophet Professional prior to the update to Prophet Enterprise. In such case related run structures were corrupted. This has been fixed.
- There was an issue with updating basis documents with the Assumptions Manager API if there were links to Input Data – Excel Spreadsheets. This has been fixed and the changes are applied with the Set-BasisDocument command.
- The performance of workspace import has been improved. The benefit should be observed mostly for environments with large base sets data.
- The performance of submitting Assumptions Manager job definitions has been significantly improved. The change was done for better handling submission of multiple job definitions at a time.
- Assumptions Manager job definitions will now open much quicker in read only mode. With this change only necessary information is being called on job definitions opening.
- There was an issue with persistency of the AoC check box in Assumptions Manager job definitions on backup and restore operations. This is now fixed and the check box selection is correctly restored or promoted.
- An issue where metadata in the FRAPIResultsSource.GetRunNumberList() method was returning default values for jobs with a failed status has been fixed so that data is returned if available.
- An issue has been fixed for nested structure jobs with an inner structure split by simulation run setting where the machines needed for the inner structure were being released prematurely as they weren’t required for the outer structure, resulting in the inner structure using fewer machines than requested.
Prophet Enterprise 2021 Q2 Service Pack 1
- The Prophet Enterprise Technical Guide has been updated to describe how to perform an upgrade of the Prophet Enterprise environment. This section was not present in the documentation for Release 2021 Q2.
Prophet Enterprise 2021 Q2
Performance improvements
- The base set entity data code has been improved so that the latest version of base set tables is returned. It was previously returning all versions.
- Run settings will open quicker as the validation of base set links has been improved.
- The basis document update has been improved by only checking the uniqueness of the entity name against the original name stored in the basis document data, instead of all entities from the database. n Finished job and job templates now open faster.
- The performance of duplicating a run structure has been improved.
Backup/Restore and Promote
- An issue whereby users could not restore a large base set from a backup file has been fixed. This issue was found when restoring a base set with a few hundred tables where each table had lots of versions.
- After promoting a workspace from one environment to another there was an issue with comments in the audit log being removed or blanked out. This has now been fixed.
- There was an issue when if a user cancelled a backup, the backup would incorrectly continue. This has been addressed and the backup is now stopped. A similar issue where an object restoration with the Execute on Server option enabled could not be cancelled has also been addressed. The cancel operation is now properly logged in the domain log.
- Previously, when taking a backup of any Prophet Enterprise object, all the categories which could be applied to that object would be automatically included in the backup. This was the case for all categories regardless of whether their values were assigned to the object or not. This has been fixed so that now only categories which are assigned to an object are included in the backup.
- Promoting more than one dataset at once resulted in only one completing successfully. This has been fixed.
User interface
- Trailing slashes in the Web Service URL when logging into Prophet Enterprise previously caused an Error in web service call which has now been fixed.
- If a user specified more than one parameter in the job template then only the first parameter would be shown in the Parameters tab when the user selected the Show only parameters that are in use check box. This has now been fixed.
- If a parameter took a value that has a file extension (such as a table name), when the Show only parameters that are in use check box was selected the parameter would not appear in the list. This has been fixed.
- An issue whereby changes to the Text Results selection in the run setting was not saved if the user exited Edit mode has been fixed.
- Previously a user was unable to view stochastic summary results from within Prophet Enterprise Studio because the Stochastic Summary result type was not available on a dropdown list. This has been fixed.
- There was an issue with the Disable Multi-Threading information shown when your job used parallel model point processing (PMPP). When a user selected a job from the Active Jobs list, the Properties pane was displaying Disable Multi-Threading as True instead of False. This has been fixed.
- In previous versions a user would get an error message when clicking View Results on a cancelled job, even though there are no results for a cancelled job. The issue has been fixed so the error no longer appears on clicking View Results but you are unable to run any query.
- If a user created a version or reverted to a version on a workspace with structure links, those structure links would be missing. This has been fixed.
- If a user used the Export Results functionality to export results and subsequently reimported those results using the Import Results functionality, opening the runlog would fail. This issue has been fixed.
- An issue where, after upgrading Prophet Enterprise from 2018 Q3 system version or earlier, existing runlogs could not be opened from Prophet Studio has been addressed.
- A workspace would not upload into Prophet Enterprise if it contained structure links that were not used by any of the run structures. This has been fixed.
- Users could not see data sets in the Prophet Enterprise Explorer, even if this view was selected for their role. This issue has now been fixed.
- There was an issue with reimporting a workspace such that if a user removed any accumulations in Prophet Professional but they were still selected in some run structures in Prophet Enterprise, the workspace import failed if the structures were not included in the workspace import. This has now been addressed.
- An issue with changing compartments for a workspace, which references any base set, has now been fixed.
- An issue opening aggregation results in External Diagram View such that the results type was left blank has now been resolved.
Prophet Enterprise API
- Model point batch size should only be within the range of 1 – 1,000,000. If a user tried to set the value outside of this range in the Prophet Enterprise API then the user would not be informed of that and the API would just assign the range boundary value automatically. In this release, if a user assigns a model point batch size outside the permitted range, a warning message is triggered and the previous value is kept.
- An issue where a user was not able to submit a job with integer numeric parameter values has now been fixed.
- An issue where a user was not able to overwrite parameter values in a job template has been addressed.
- A small enhancement has been made to the Prophet Enterprise API so that a user will be able to upload all files under a workspace via the API.
Push to PE
- Changing a machine group for a queued Push to PE job would cause the job to fail. This has been fixed.
- There was an issue with the Push to PE feature such that when submitting multiple run pairs, the second and subsequent job run pair would fail. This issue has been resolved, with the following limitations in Push to PE run pairs. For a specific workspace:
- Run structures must be unique.
- Run numbers must be unique.
The preparation stage for each of the run pairs is performed sequentially rather than in parallel. The runtime of multiple run pairs can be performed in parallel. You cannot suspend and resume a Push to PE job. Instead, cancel the job and resubmit it once the other jobs for that workspace have finished.
Calculation engine
- A couple of issues obtaining the results metadata of products in certain types of Prophet Enterprise run, such as when using Copy Results functionality where the destination already has results in it, has been fixed. The issue caused the results to be unreadable for affected products and affected both READ_RESULTS during a run and Excel Reporting.
- When submitting a job with a READ_RESULTS call reading from an external results location, where those external results were produced using copy results the read would fail in the case where the copy results location contained results for multiple jobs with different variables in the results data. This has been addressed.
- In the 2019 Q2 and 2020 Q2 releases, if you used a parameter to name your run number folders in combination with the copy results functionality then pre-existing folders in the specified location would be overwritten, even if different parameter values for the run number folders were used. This has been fixed.
- An issue where PMPP runs could fail due to a clash between the calculation threads trying to read results metadata and the thread outputting finalised spcode results has been addressed. The issue was first noticed in a job using READ_FLEXIBLE_TABLE.
- A bug where runs using the Calculated variables run structure option, in conjunction with Table and MP File Optimisations (TMPO) set to Full, could erroneously produce missing DLL errors has been fixed. This was due to the Same As products trying to load a DLL with the name of the base product despite TMPO=Full runs producing product DLLs for each Same As product and thus not producing a base product DLL.
- An issue in which selecting Only run first model point for each Sub-Product Code on the run setting did not always persist their values between dynamic loop iterations in a PMPP run has been addressed.
- A bug where accumulations could not be generated for a Prophet Enterprise job, where the accumulation read from model components in multiple dependent jobs, has been fixed.
- There was an issue when writing results for split by simulation runs to ArangoDB, where each simulation could write out the same unit prices file, which caused a conflict on writing results. This issue has now been resolved.
- For a job with read functions – PROJ_RESULT, STOCH_RESULT or AGG_PROJ_RESULT – the read would unexpectedly fail if the path in the external results location provided in the run setting included the run number folder. In the case where the run number folder matched the run number specified as a parameter value in the read function, the read would fail. This issue has now been resolved.
- In the 2019 Q2 and 2020 Q2 releases, if a job was reading from an external results location (provided in the run setting) using either a READ_RESULTS function or an accumulation, the run number from which the value was read from would differ depending on the system version used to produce the original results. Both READ_ RESULTS and accumulations have been corrected to always read from the run number corresponding to the run number being processed in the current job, regardless of the results version.
- In the 2019 Q2 and 2020 Q2 releases, the time taken to create the zip file when the create a results file for download option was enabled could be noticeably longer than in earlier releases. This has been resolved by changing the compression library used and users can now expect the post-job zipping time to be as good as, if not better than, previous levels.
- A bug in which reading outside array bounds failed in descending optimised loops for 2-dimensional arrays has been fixed so that zeroes are returned outside the array bounds.
- A bug that could cause structured exceptions when an extended formula array resized while being read has been fixed.
- A bug has been fixed where some looped module inputs, that vary by loop iteration and are used directly in IF conditions, were not updated for each loop iteration within their dependent module formulae.
- The error handling of the ARRAY_MAX function has been improved to prevent structured exceptions in some cases, and make clearer where the syntax is incorrect in other cases.
- A syntax-checking bug has been fixed that caused incorrect formulae, with respect to data types and comparisons, to not be caught properly. This resulted in malformed C++ code generation and a “merged code error” with C++ compiler messages that would not be understandable in terms of the underlying fault in the model’s definition.
- A t-limiting bug has been fixed that caused errors involving uncalculated values and out of bound time references to be masked and zeroes silently used instead.
- A t-limiting issue affecting t-independent extended formula, sequenced in the same calculation group as variables that are not t-limited, has been fixed. This issue had caused extended formulae variables to appear in the DGN but not actually be executed, and was observed due to incorrectly ignored breakpoints in the Prophet Debugger. Perturbing the sequencing by adding in references to the affected extended formula had been the only workaround prior to the fix.
- An update to the 2017 and 2019 compilers, to fix a C++ defect, caused existing Prophet code generation to start failing. Prophet code generation and compiler flags have been altered to comply with the new rules.
- The validation check that ensured that each data row in a model point file had the same number of columns as the header, after allowing for treating consecutive column delimiters as one column delimiter, was only working on the first data row. The check has now been fixed for all the model point data rows.
- An inefficiency that caused empty or zero filled results files to be produced for unexecuted products, when a run is aborted, has been fixed.
- An issue where lots of small files and empty subfolders were left behind in the Prophet temporary directory after a run finished has been fixed.
- We have addressed an inefficiency in the handling of dynamic debug results files in order to reduce the number of unnecessarily repetitive file operations. In addition, the code for creating a sub-directory, whose parent folder(s) may not exist yet, has been optimised to further reduce file operations.
- The logic for moving or copying a file over the top of an existing file was inefficient and could fail if the file server did not delete the existing file in a timely fashion, for example due to anti-virus scanning. This issue has been addressed.
- A performance inefficiency in READ_RESULTS has been fixed that particularly affected products that read a lot of results such as Summary products or ALS External.
- An inefficiency in the results formats introduced in Release 2019 Q2, that caused the results to include unnecessary data repetitively, has been addressed. The issue was most noticeable when using stochastic results on the file system where the number of variables and time periods per simulation was low. However, the issue still represented an inefficiency for other results types such as projection, especially when variables and time periods were heavily trimmed by the user.
- An issue causing runs to display “invalid date/time string” warnings in the run monitor has been addressed. The issue was compiler-related and only affected certain regional settings in a non-deterministic manner.
- In the previous release, if you changed the selected variable group on an aggregation results definition post-run then every product selected in the definition was regenerated. This has been fixed such that only the products affected by that change in variable group are regenerated.
- An issue where certain results types, when using the file system for results storage, are appended to for each dynamic iteration rather than replaced, has been addressed. This caused certain results to grow significantly bigger than pre-9.1 results. To fix this issue, t-independent variables are only output in the first dynamic loop for aproj, astoch, pproj and pstoch. This also prevents additional issues where t-independent cumulative variables, including ones that do not vary by dynamic loop start date, effectively reduced significantly as policies expired across dynamic loop iterations, and where the sum across aggregation values might not agree with the totals when one or more aggregation values ended up with no more active policies left to process prior to the end of the dynamic loop. These latter issues still exist for legacy results types, with respect to spcodes, such as projection and stochastic.
- An inefficiency with the READ_RESULTS function was introduced in Release 2019 Q2 where the Calculation Engine (CE) checked the results frequently to identify whether a requested result was part of a product or accumulation. This involved checking whether the requested product or accumulation was part of the currently executing run pair, and if not, checking whether it was part of the pre-existing workspace results, which is relatively expensive in terms of run time. Performing this check repeatedly for every READ_RESULTS call could significantly impact the runtime of products using lots of READ_RESULTS calls, for example Summary library products. If the workspace was located on a remote location to the worker(s) then the performance impact could be more adverse. This issue affected models where a lot of products and accumulations results being accessed were either missing or not part of in the external results location. This issue has now been fixed.
- Prophet 2019 Q2 introduced an issue in which all variables for a particular set of variable-by-time results were loaded into the results cache when a single variable was requested. The issue only affected the CE results-reading runtime functions PROJ_RESULT and STOCH_RESULT and the Excel Reporting worksheet functions of the same name. This could cause a significant memory increase during runs which access a small subset of variables across a large number of these sets, for example a summary product accessing a small set of variables across a large number of products and sub product codes using the affected functions. This issue has now been addressed for results stored using the file system directly, but not for results stored using ArangoDB.
- There was an issue with sensitivity documents if the sensitivity value for an assumption was equal to the value defined in the related basis document, and the value in the basis document was defined to be different from the underlying table value. Under such conditions the sensitivity resulted in the underlying table value being used instead of the value defined in the basis document. The issue was present for both cases, when applying sensitivity to the reporting date and to a specific AoC step. This has now been fixed.
- Two threading issues in the Prophet 2020 Q2 Calculation Engine that affect PMPP have now been fixed:
- If using PMPP and READ_FLEXIBLE_TABLE it was possible to get a putResults error of the type “concurrent or
recursive” calls.
- If using PMPP and modules it was possible to get a structured exception when two DLLs both tried to load the same table at the same time. Depending on whether fast runtime or fast compile was used in the structure, or whether a module link was set to be looped, a DLL could represent a single module shared amongst module links in the same product or individual module links.
Grid services
- Scheduler and Monitor Manager services in Prophet Enterprise will now show messages in their log files to record when they are started. This can help identify when they have been restarted either deliberately or by the operating system due to a fatal error.
- The configuration file – system.config.xml – was not accessible from Prophet Grid Administrator (PEG Admin). This issue has been fixed so that the configuration file is now editable.
Licensing
- Validation has been inserted to prevent detached licence users from changing their Client Licence Server (CLS) configuration details in Prophet Licensing, as this renders the licence unusable.
- At CLS installation, in cases where startup of the CLS Windows service took longer than 90 seconds, the installation could fail with error code 1920. A change to the CLS installer should mean this error no longer appears.
- Robustness of the CLS Windows service has been improved by removing a database shrink operation.
- Email verification in the CLS has been improved such that you can no longer add duplicate user email addresses to the same group licence.
- There was an issue recording licence consumptions in the CLS portal when the CLS machine’s disk was full. This has been fixed so that now if licence acquisition works then the consumption is correctly rendered in the CLS portal.
- Some settings changes in the CLS require a restart of the CLS Windows service to take effect. However the message informing CLS users of this was easy to miss. The instruction to restart is now clearer.
Plugins
- An inefficiency in the Intex plugin has been fixed so that we no longer request the underlying Intex API produced output that can’t be accessed by a call to the Intex plugin.
- A missing ADCO DLL that was only needed when using ADCO implicitly with Intex, via the INTEX.DEAL function call, has been added back into ADCO Plugin version 2.2.0.
- Intex has dropped support for calling ADCO directly from within their software to get prepayment rates. This functionality no longer works and resulted in a structured exception in the Intex plugin for versions 3.4g onwards. The Intex plugin now does a check to error gracefully if you try to use the ADCO prepayment method when calling INTEX.DEAL. Version 3.4f_p2_sp1 is the last Intex plugin to support this method. We are working on an alternative approach for a future release.
- A bug has been fixed in the Intex plugin affecting the PAY_TO_PAY function when requesting certain results types such as WAC, WALA or collateral balance ratios. The problem was that a weighted average calculation did not divide by the sum of the weights at the end.
Excel Reporting
- An issue involving the time taken for Excel Reporting to open projection results has been fixed.
- In previous versions, on the Prophet tab of the ribbon in Excel, if you clicked Options and then cleared the default selection of Show Variable Errors, an “unset nullable” error was returned instead of zero. This issue has now been resolved.
- An issue has been fixed involving Excel Reporting being unable to get metadata for Same As products in Prophet Enterprise copy results folders when folders contain existing results.
- An issue has been fixed involving Excel Reporting queries not executing properly if worksheet functions were called first and the Results subfolder was passed in as the result store location to the query.
- There was an issue involving the time taken for Excel Reporting to respond when unable to connect to ArangoDB. This issue has been fixed.
- An error in Excel used to appear when using the Results_Date_Time function to obtain unit prices results in ArangoDB. This is no longer an issue.
|
2020 Q2 |
Flexible Results
For Release 2020 Q2 Service Pack 3, two results types have been made available for users licensed for Flexible Results: aggregation results (which are new) and individual policy results. These results types give users added flexibility in defining and generating results and are both supported by the ArangoDB output location added in the last release.
Aggregation results
Aggregation results allow you to generate projection results aggregated by a model point variable of your choosing, called the aggregation variable. Aggregation projection results can be created for up to two aggregation variables per product in addition to existing sub-product aggregation functionality.
Aggregation results are specified in a new workspace object, the Aggregation Results Definition, before adding to the run setting in the Flexible Results section of the Results tab.

The new results definition can also be created, edited and added to your job’s run settings via the API.
Aggregation results output can be viewed directly in the Prophet Studio application, Results Viewer and with Excel Reporting through a new AGG_PROJ_RESULT function.
Individual policy results
Individual policy results were previously only available to clients with a Production Interface licence, they are now also available to users licensed for Flexible Results. Individual policy results allow you to create projection or stochastic output for specified policies in your model point file, up to a 100 policy limit.
The specification of these results adopts a similar approach to aggregation results, via a workspace object which can then be referenced by multiple run settings. Individual policy results can also be set up via the API, read directly in the Prophet Studio application, Results Viewer and with Excel Reporting through a new POLICY_RESULT function.
For further information on either aggregation or individual policy results, see the Prophet Professional 2020 Q2 Bulletin.
Flexible Results API
The Flexible Results API (FR API), which was released in 2020 Q1 for Prophet Enterprise users, has been updated to read both aggregation and individual policy results. The FR API is a .NET Dynamic Linked Library (DLL) that can be used by a bespoke application to directly read Prophet results at scale without the need to first load the data into the Prophet Results Database (PRD).
The Insurance Data Repository (IDR) can now use FR API to efficiently extract, transform and load Prophet results data with minimal changes to any existing jobs that were using PRD as a data source.
Table and model point optimisations with modules
Table and model point optimisations can now be used with products containing modules. Table and model point optimisations can be enabled on the Runtime tab of the job or job template.
For further enhancements made to modules as part of this release, see the Prophet Professional 2020 Q2 Bulletin.
Usability enhancements
- Improvements have been made to speed up the access and display of results in the Prophet Enterprise user interface.
- The performance of opening run settings when they reference base sets in the tables tab has been improved.
- Domain settings have been split between two permissions: Administrative Tasks and Actuarial Administration. The new Actuarial Administration permission grants access to non-system administrator permissions like the ability to amend table locations, job priorities or default upload options. The change does not affect existing roles but does enable custom roles to be more specifically tailored to your environment’s needs.
- Monitor Manager can now update itself when the list of machines in a grid changes without needing to be restarted. It does this by polling the list of machines, and when it detects any changes it adds and removes machines as necessary. The polling rate is configurable but by default is set to 0 so that no polling happens and the original Monitor Manager behaviour is preserved.
|
Prophet Enterprise 2020 Q2 Service Pack 3
- There was an issue with the Push to PE feature so that when submitting multiple run pairs, the second and subsequent job run pair would fail.
This issue has been resolved, with the following limitations in Push to PE run pairs. For a specific workspace:
- Run structures must be unique.
- Run numbers must be unique.
The preparation stage for each of the run pairs is performed sequentially rather than in parallel. The runtime of multiple run pairs can be performed in parallel.
You cannot suspend and resume a Push to PE job. Instead, cancel the job and resubmit it once the other jobs for that workspace have finished.
- The Prophet Enterprise functionality that lets a user copy results to an output location, and potentially merge them by run number with existing results in that location, did not correctly work if the location had existing results in it. In such situations, results viewing components such as Excel Reporting could not report on SameAs products in the results: an error, relating to not finding a key piece of metadata required to interpret the results files, was returned instead of the requested results. This issue has now been fixed.
- An inefficiency with the READ_RESULTS function was introduced in Release 2019 Q2 where the Calculation Engine (CE) checked the results frequently to identify whether a requested result was part of a product or accumulation. This involved checking whether the requested product or accumulation was part of the currently executing job, and if not, checking whether it was part of the configured external results location(s), which is relatively expensive in terms of run time. Performing this check repeatedly for every READ_RESULTS call could significantly impact the runtime of products using lots of READ_RESULTS calls, for example Summary library products. If the results were located on a remote location to the worker(s) then the performance impact could be more adverse. This issue affected models where there was a valid external results location configured and a lot of products and accumulations results being accessed were either missing or in the external results location. This issue has now been fixed.
- In previous versions, on the Prophet tab of the ribbon in Excel, if you clicked Options and then cleared the default selection of Show Variable Errors, an “unset nullable” error was returned instead of zero. This issue has now been resolved.
- Prophet 2019 Q2 introduced an issue in which all variables for a particular set of variable-by-time results are loaded into the results cache when a single variable is requested. The issue only affects the CE results-reading runtime functions PROJ_RESULT and STOCH_RESULT and the Excel Reporting worksheet functions of the same name. This can cause a significant memory increase during runs which access a small subset of variables across a large number of these sets, for example a summary product accessing a small set of variables across a large number of products and sub product codes using the affected functions. This issue has now been addressed for results stored using the file system directly but not for results stored using ArangoDB.
- There was an issue with sensitivity documents if the sensitivity value for an assumption was equal to the value defined in the related basis document, and the value in the basis document was defined to be different from the underlying table value. Under such conditions the sensitivity resulted in the underlying table value being used instead of the value defined in the basis document. The issue was present for both cases, when applying sensitivity to the reporting date and to a specific AoC step. This now has been fixed.
Prophet Enterprise 2020 Q2 Service Pack 1
- This update fixes two threading issues in the Prophet 2020 Q2 Calculation Engine, that affect PMPP:
- If using PMPP and READ_FLEXIBLE_TABLE it is possible to get a putResults error of the type “concurrent or recursive” calls.
- If using PMPP and modules it is possible to get a structured exception when two DLLs both try to load the same table at the same time. Depending on whether fast runtime or fast compile is used in the structure, or whether a module link is set to be looped, a DLL may represent a single module shared amongst module links in the same product or individual module links.
Note: It is recommended that users apply this update if they are using PMPP.
Prophet Enterprise 2020 Q2
- There was an issue with erroneous generation of base sets when referenced directly in a run setting as part of a job. The issue generated a version for the base set, regardless of whether a base set change had been made since the last version was taken. This would then fill the database with many copies of the same base set version, which affected performance. This has been fixed so that now versions are only created when the base set data is unique when submitting a job. This is now consistent with other methods of job submission that use base sets.
- There was a performance issue that caused file system writing for certain types of results files to bypass the Windows file cache, causing some Prophet models to run noticeably slower. This has now been fixed.
- An issue which meant copying run settings from one workspace to another would, on occasion, throw an error has been fixed.
- There was an issue with the Override Results Directory names option which meant multiple run numbers could be stored in the same directory. Validation in the run setting now prevents this from occurring.
- Bugs in the Prophet Enterprise Administration Tool (PEAT), found when changing Data Storage from By Compartment to By Domain and vice versa, have now been fixed.
- There was a bug in the Prophet Grid Administrator which meant a spurious error message would appear if your web server address included a port number. This has now been fixed.
- The Prophet Enterprise runlog has been made more resilient to instances where the ArangoDB server goes down in the middle of job preparation. In this scenario, accurate start and finish times for preparation are now shown.
Prophet Results Database
- In Prophet Enterprise Release 2019 Q2, the Create a results file for download option would create a zip at a different level in the results file structure compared to previous releases. This meant that Prophet Results Database uploads using the -cri argument would not succeed. This has been fixed so that the results are once again zipped at the pre-2019 Q2 level.
|
2019 Q2 |
Flexible Results: Distributed Database Output
For those users who are licensed with the Flexible Results feature, 2019 Q2 provides users with the ability to output results to a supported distributed database providing significant improvements in performance on large, scalable runs.For some models, the speed of reading and writing results is the crucial factor affecting execution times. The windows file system technology traditionally used by Prophet can form a bottleneck for these models. As data volumes increase (because of regulatory pressures such as IFRS 17), the number of models affected by this also increases, and the execution time becomes an important issue. The distributed database technology at the heart of the Flexible Results module supports optimised disk usage, parallel writing and file compression, allowing users to address this bottleneck, and giving them greater control over the results data produced by the models.
Larger models benefit from the scalability achieved through the faster writing and better organisation of the results output, without relying on expensive specialised hardware.
In the 2019 Q2 release, users licensed for Flexible Results can configure Prophet Enterprise to output results to ArangoDB. ArangoDB is a NoSQL database system supporting parallel writing and file compression, and can be installed from the Prophet Enterprise installation disc. For this release ArangoDB must be installed on a Linux operating system. This is due to the greater
performance achieved on Linux. More information on ArangoDB can be found in the Prophet Enterprise Technical Guide, and on their website at https://www.arangodb.com/.

A new tab has been added to the Prophet Enterprise Administration Tool to define Arango configurations. These configurations
detail which database and server the configuration applies to, and how workers connect to the specified database when jobs are
submitted.

The data storage setup tab has also been updated to allow users to choose a results storage mechanism. Users licensed for Flexible Results can either choose to output results to the Prophet Enterprise file system as normal, or to select one of the previously defined Arango configurations. This configuration is applied at a domain level, so different domains can be set up to output to different Arango databases. This will apply to all future jobs submitted after the configuration is set, providing users the flexibility to switch between storage locations over time.

Text results and runlog information will continue to be stored under the job folder on the Prophet Enterprise file system, even if the other results are stored in Arango.
Results change
To improve the scalability of Prophet under our Flexible Results project, enhancements have been made to how results are stored in Prophet Enterprise. Flexible Results is a new licensable feature for Prophet Enterprise users, however these changes will be seen by all users including those who are not licensed for the Flexible Results feature.
To optimise these changes the names of the files stored in the results folder have been altered. It should be noted, however, that this has no effect on how results are presented in the Prophet Enterprise user interface, and the same names for the results files used previously are retained here for your convenience. Results produced by Prophet Enterprise can still be used with other Prophet applications and you can still view results produced by previous versions of Prophet in this latest release.
Looped Modules
Prophet Enterprise 2019 Q2 supports the looped modules functionality introduced in Prophet Professional 2019 Q2. Looped modules workspaces created in Prophet Professional can be uploaded to Prophet Enterprise. For more information on Looped Modules please see the Prophet Professional 2019 Q2 Bulletin.
If you are interested in seeing how Modules and Looped Modules can be used please speak to your Sales Representative or refer to the user manuals.
Advanced Vector Extensions (AVX)
Looped Modules works best in conjunction with Advanced Vector Extensions, which allows the loops of the Looped Module to be run in parallel. AVX is a set of CPU instructions that enable vectors of integers, singles or doubles to be processed simultaneously. This allows the processor to calculate a lower number instructions on the same amount of data and thus increase
performance. This will be of particular benefit to cloud users, for whom runtime directly affects costs.
Each worker process is now capable of executing different Single Instruction Multiple Data (SIMD) instructions concurrently. For example, the following four instructions might be sent to the CPU:
18 + 11
12 + 5
6 + 8
32 + 2
Traditionally, this is processed by the CPU as four separate instructions which must be undertaken one at a time (ignoring complexities around the difference between Floating Point Unit (FPU) and Arithmetic Logical Unit (ALU) processing and how these can be parallelised). With AVX, this set of instructions could be processed simultaneously, by refactoring this set of
instructions as one vector addition:

Performance is fundamentally driven by how many instructions the CPU can perform per second. By reducing the number of instructions required to process the same data, AVX can significantly increase performance.
AVX is licensable functionality, and you must also have a Prophet Enterprise licence. Your model must also make use of the Looped Modules functionality as it is the loops that are optimised to use the AVX functionality.
If you are interested in finding out more about AVX and how it can help then please speak to your Sales Representative.
Prophet Grid Administrator API
We have developed a new API for use with Prophet Grid Administrator. This API enables the use of the key functionality from Prophet Grid Administrator via an API, for example:
- Adding/deleting machines from a grid
- Adding/deleting machines from domains
- Adding/deleting/editing machine groups
- Adding/removing machines from machine groups
- Adding/removing machine groups from compartments
This API is accessible via PowerShell or .NET, and so is able to be used by Prophet Control Centre, or whichever mechanism you currently use to access our existing APIs.
Improved data transfer protocols
We are always seeking to improve the security, reliability, and speed of our software. To that end, for 2019 Q2 we have deprecated BITS and moved to WCF chunking as our data transfer mechanism.
- Upload and download of data from Prophet Enterprise servers are now handled by WCF.
- References to BITS in the Studio UI and configuration files have been removed.
Size of the batch can be specified for split by model point batch
The Model Point Batches split type has been improved. In the run setting you can now specify the number of model points that each batch contains rather than having to use the default of 10,000. This provides flexibility to tune your model performance.
Categories of finished jobs
Finished jobs were previously read-only entities. This could cause administrative issues when new categories were created in the Prophet Enterprise environment. From Release 2019 Q2 you are able to change the category of finished jobs. The rest of the entity remains read-only to protect the audit trail.
Improved security
You are now able to configure the SMTP mail server to send emails using SSL for notifications.
We have additionally made other security enhancements, which are invisible to end users.
Streamline machine entry
In previous releases you could use the Monitor Manager config file as a mechanism to add machines to your Prophet Enterprise grid. This approach is no longer supported and has been changed to simplify machine administration.
Improved error logging
Some error logging has been improved for the API, particularly with regard to serialisation errors.
Note: From this release we no longer support 32-bit compilation of Prophet code and calculations need to be performed using the 64-bit runtime architecture. Infrastructure that only supports 32-bit will not be able to be used with Prophet Release 2019 Q2.
|
- A bug which sometimes prevented you from accessing the runlog via the web service when using HPC has been fixed
- A bug which caused time-outs when filtering a very large list of finished jobs that were using many parameters has been fixed
- A bug which caused the scroll bar to not appear on the categories and parameters window of the job creation wizard has been fixed
- An issue which meant there could be duplicate email addresses in the Notification tab of the Job Submission dialog has been fixed
- A user interface issue with nested structures jobs which meant the same structure was duplicated in the versions dialog box has been fixed
- A bug which sometimes caused the runlog to not be indexed after resubmitting a failed job has been fixed
- A bug with the Allow Concurrent Model Point Batches check box in a product which meant that the correct value could be lost on creating a new version of a workspace has been fixed
- Having an expired password on the target environment when performing a promote caused an error. This has been fixed and users are now correctly prompted to reset their password.
- An issue with reconnection via the API after a session is lost has been fixed
- Some issues with errors produced during a database upgrade have been fixed
|