Skip to content

Data Conversion System (DCS)


The Data Conversion System (DCS) is an application which enables you to convert the contents and format of data stored in a range of formats: Prophet and Glean files, fixed (ASCII), delimited, Excel and database extract files. DCS can produce per-policy output files for use in individual policy Prophet calculation runs or as input into Prophet data grouping runs. You can also use DCS to produce grouped output files for use in grouped data Prophet calculation runs. In addition, DCS can create text files (with the output fixed width or delimited format), ODBC format files and comma delimited files in Prophet’s model point file format. By creating more than one output file and/or format from a program, you can produce an output file per product or product class (that is, per Prophet product specification). You can also create test model point files without any source data.

UpdateBulletinBest PracticeProcess ControlReconciliation
The following enhancements have been made in each DCS release.

Release Enhancement
2022 Q1
2021 Q2
2020 Q2
2019 Q2
  • Improved copy-paste functionality in the run settings
  • Support for Microsoft Access Database Engine 2016 redistributable
  • Improvement to performance of table reads
2018 Q3
  • New licensing solution
9.0.4
  • New functions added
  • Generic table functions now support absolute file paths
  • Options dialog improved
  • Debugger enhancements
9.0.3
  • OUTPUT_ARRAY_AS_CONSTANTS enhancements
  • Significant increase to DCS limits
  • Improved table creation
9.0.2
  • Copying input and output formats
  • Nested structures grouping improvements
9.0.1
  • DCS Plugins
  • Nested Structures – Grouping from Prophet Professional
9.0.1
  • DCS Plugins
  • Nested Structures – Grouping from Prophet Professional
9.0
  • Improved rounding tolerances
  • New functions
8.2.3
  • Improved debugger, including conditional breakpoints
  • Improved saving functionality
The enhancements and addressed issues in each DCS release are summarized below.

Release Enhancement Addressed Issue
2022 Q1
2021 Q2
2020 Q2
  • An issue where mismatching capitalisation of text variables in the Code and Output format would result in Grouping errors has been fixed.
  • An issue with locking of Access Database output files has been fixed.
2019 Q2 Improved copy-paste functionality in the run settings
Support for copy, cut and paste options in the user interface has been improved. Common keyboard shortcuts for these operations and accompanying clipboard options in the ribbon now work for all text fields in the run settings.

Support for Microsoft Access Database Engine 2016 redistributable
We now support the use of the Microsoft Access Database Engine 2016 in DCS, so that Microsoft Access 2016 and Microsoft Excel 2016 can be used as input and output data sources, with the latest performance enhancements and bug fixes.

Improvement to performance of table reads
The underlying code for READ_GENERIC_TABLE statements has been modified so that performance of DCS scripts with large numbers of such calls can see improvements. Note that performance will vary across workspaces and is dependent on a number of factors including hardware and compiler used.

  • An issue which prevented scripts from outputting to both GDFx and ODBC file formats in the same run has been fixed.
  • The compilation log sometimes included a lot of white space in Windows 10. This has now been addressed and no white space appears.
  • An issue where the NUMVAL function incorrectly reported an overflow/underflow error has been fixed.
  • Boundary validation on integers and short integers in the input format has been corrected to catch overflow and underflow errors.
  • DCS now correctly reports all tables used within the script.
2018 Q3 Licensing
Prophet Release 2018 Q3 introduces a new licensing technology. The previous licensing solution will not be supported with any release from Prophet Release 2018 Q3 onwards. Pre Release 2018 Q3 Prophet versions will continue to work and will be supported following the standard support lifecycle. However if you require new licence keys for additional users or due to hardware replacement then this will not be possible after December 2020. We therefore advise that all clients upgrade to Prophet Release 2018 Q3 or later well in advance of December 2020 in order to avoid any down time. Please contact your account manager to discuss how we can be of help. Unlike previous versions of Prophet this licensing framework will be written and controlled by FIS. This has some major advantages including simpler client installations and more rapid response to issues. Licences will now be user driven rather than machine. This will help both clients and FIS to implement the solution much more quickly, when new machines are added or replaced in an environment or when a disaster scenario occurs. We will also be able to expand the functionality beyond the restrictions imposed previously. Other advantages include better auditability and tracking of licences and improved support for virtualised environments. The licence options for Prophet Release 2018 Q3 are:

  • User licences: Tied to an individual user’s Active Directory account such that only the Prophet user with the email specified on the licence can acquire it. This is similar to the previous Machine-locked solution.
  • Group licences: These are licences that can be used by multiple users (from a defined list) within an Active Directory group, as controlled by their email addresses and selected by the customer. This is similar to the previous Floating licence.
  • Site licences: These are licences that can be obtained by anyone who has access to the client licence server that is distributing the licence. The technology is safe for virtualised environments.

New licences will be allocated as follows. Machine-locked licences will be transferred, one-to-one, into User licences. Floating licences will be transferred, one-to-one, into Group licences.

New licences will be pushed out by us to your Client Licence Server (CLS). The CLS synchronises with the FIS Licence Server (FLS). The CLS therefore requires access to the internet and individual machines require access to the CLS. If a disaster event occurs and the CLS is lost then a new CLS simply connects to the FLS where replacement licences are synced.

If you require more advice or support regarding the changes to the licence solution please contact FIS Client Services or your local FIS representative.

Installation location
Prophet installed files are now placed in C:\Program Files (x86)\ProphetSuite, C:\Program Files\ProphetSuite and C:\ProgramData\ProphetSuite or equivalent.

Calculations

  • Some issues around how plugins interact with the runlog and DCS script have been fixed.
  • The debugger would become unresponsive when trying to debug an EBCDIC workspace. This has been fixed.
  • Copying a workspace that uses a DRP file could cause the run to fail as it would still try to use the original location. DCS has been changed to use the DRP file in the location of the runtime executable.
  • DCS would crash if input fields had more than 36 characters. This has been fixed.
  • When using the 64 bit 2015 compiler the debugger may crash whilst running. This has been addressed.
  • An issue when that caused the DCS run to fail using 64 bit with Excel output has been fixed.
  • Issues with additional format records and ODBC input caused double records to be outputted. This has been fixed.

User interface
Improvements and fixes to the user interface include:

  • Attempting to expand the program node by double clicking causes DCS to try to open it as a table in Prophet Viewer.
  • An exception was previously given when using DCS Compare if the user tries to go to a line beyond the end of the file. This has been fixed.
  • DCS would previously crash when attempting to use the Insert Variable menu. This has been fixed.
  • When using an xls file as an input DCS would fail to open the input record when prompted. This has been addressed.
  • An error produced when creating a new input format for a new DCS workspace has been fixed.
  • Scrolling in the output format would cause the variables to be grouped to be checked. This has been fixed.
9.0.4 Calculations
New functions have been added to DCS which enable the reading of tables relative to the executable location:

  • GET_EXECUTABLE_LOCATION: Returns a text value containing the full path of the current DCS program executable.
  • GET_CURRENT_OUTPUT_FILE_PATH: Returns a text value containing the full path of the output file location.

The following functions now support absolute table paths as well as relative:

  • GENERIC_TABLE_CLOSE
  • GENERIC_TABLE_EXISTS
  • READ_GENERIC_TABLE
  • READ_GENERIC_TABLE_TEXT

User Interface
The settings in the Options dialog box have been grouped by Workspace Options, User Options and Machine Options to make it more consistent with other Prophet applications.

The DCS program name has been added to the title bar and taskbar so that you can distinguish between programs easier.

Input and output formats now appear in alphabetical order with sub-records and additional records as child nodes of their parent main format.

Debugger
The DCS debugger has been improved to make it generally easier to use:

  • Watches and panel state remembered on exiting and re-entering debug mode.
  • Breakpoints automatically adjust when new lines of code are added in the code editor.
Calculations

  • An issue which meant that a DCS run would fail if a dynamic array was set as a sorting/grouping variable in an output format and passed into an OUTPUT_ARRAY_AS_CONSTANTS call has been resolved. This means that dynamic arrays which are text variables can now be included in the output format.
  • We have fixed an issue that caused scripts using arrays of size greater than 32768 to fail at runtime.
  • An issue where RE_START was failing with additional record input formats has been fixed.
  • The DELETE_EXISTING_OUTPUT_FILES function would not work if the program had no input files, this has been fixed.
  • Using NEXT_RECORD as part of a procedure would move out of that procedure when processing the current record rather than moving onto the next one. This has been fixed so that NEXT_RECORD now works correctly in procedures.
  • We have fixed an issue where READ_GENERIC_TABLE was not correctly respecting the “Maximum number of runtime errors before termination” value set in the Run Settings.
  • Some issues with ODBC output to multiple tables have been fixed.
  • Array out-of-bound resizing has been made more performant by using a geometric rather than arithmetic expansion algorithm.

User Interface
Improvements and fixes to the user interface include:

  • Auto-refresh of various objects (such as the results tree) following a run.
  • Debugger fixes:
    • Watches pane shows correct values for multi-dimensional arrays.
    • Panel resizing now correctly persisted on closing and reopening.
    • Programs with modules now run without issue in the debugger.
  • Modules improvements:
    • The user interface was found to have issues handling modules with long file names; file names have now been restricted to 64 characters to prevent these occurring.
    • Modules can now be opened without error from Windows Explorer.
  • Categorisation of functions in the Function Wizard has been improved and a few cases where an incorrect function description was being displayed have been resolved.
  • The compile dialog is now modal, disabling the main user interface and preventing changes to compile-time dependent program settings.
  • The compile dialog edit button, available when selecting compile errors at the end of a failed compile or when viewing the compile log, now consistently opens the code editor to the line number shown in the error.
  • The User Interface is now partially and consistently locked down during a DCS run to ensure that actions that would attempt to delete the running executable are not permitted.
  • Case-insensitive check performed on creation of input and output formats to prevent duplicates being created.
  • We have fixed an issue where clicking ‘View Formatted’ with an Excel input format open would cause DCS to error.
  • Rename Variable functionality has been improved to prevent duplicate variables being declared or unintentional merging of two variables into one.
9.0.3 OUTPUT_ARRAY_AS_CONSTANTS with dynamic size arrays
We have enhanced the function OUTPUT_ARRAY_AS_CONSTANTS such that it can be used with a dynamic sized array that can be determined in code or alternatively fed in via an input. This means that a table or model point file can now be created with dynamic sized array variables.

Increase to the number of input/output fields supported
DCS has been extended to support the import of formats with more than 255 input and output fields. The limit will depend on the data format that is used. Please note that viewing a file in DCS with a large number of fields may cause the user interface to perform slowly.

Increased text variable size limit
The maximum text variable size has been increased from 65 to 255 characters to make it consistent with Prophet.

DCS output metadata
DCS can be used to create tables for Prophet. We have improved this functionality so that tables and model point files produced by DCS will now contain the metadata that allows Prophet to recognise the type. The file will now open in Prophet without requesting the file type and will be shown in the appropriate section of the user interface.

Runlog
When using long paths or file names in DCS, these paths would be truncated in the runlog. This has been addressed to better display the runlog under these circumstances.

ODBC
We have fixed some issues that prevented DCS from using some database types via ODBC.

Fixed Size Arrays
Some further checks have been applied for fixed size arrays. This may cause some DCS scripts to start failing where they had previously succeeded. An appropriate error is given if the variable is assigned a value for an element above the declared size. Fixing the DCS script in such cases can be straight forward for users as the array can be declared with 0 size. This effectively makes the variable a dynamic sized array which can grow as required.

9.0.2 Copying input and output formats
The functionality of being able to copy input and output formats and paste them into different DCS programs has been reintroduced into 9.0 Update 2.

Nested structures grouping improvements
The functionality to call DCS from Prophet during a nested structures run has been improved in DCS 9.0 Update 2, specifically we have made updates to

  • Grouping in models that use new business
    In the previous version of DCS, if the Prophet model contained new business model points then the DCS grouping functionality would not work because of the order of processing new business in Prophet. DCS and Prophet 9.0 Update 2 have been enhanced so that the existing business model points are passed to DCS and grouped. New business model points are appended to the grouped existing business and are then passed to the inner structure to be processed.
  • Population of arrays in the DCS script
    In previous versions of DCS array variables would be populated in DCS input formats with brackets around the array index, for example,

    Array_Variable(1)

    Users had to manually convert the brackets into underscores to save the input format. DCS population of input formats has been improved so that the conversion of array index brackets to underscore characters is done automatically.

    For multi-dimensional arrays the commas separating the indices are also converted into underscores, for example,

    Array_Variable(1,1,3)

    would be populated in the input format as

    Array_Variable_1_1_3_


    Note: This has been done for nested structures grouping input formats as well as all standard DCS input formats.


    In addition to the input being auto-populated for array variables DCS grouping scripts are also auto-populated with code to convert the inputs into a DCS array and use the DCS OUTPUT_ARRAY_AS_CONSTANTS formula to output the array back to the Prophet model, for example if you had an array in your model point file with 3 indices then the following code would be automatically populated in the Code of your DCS grouping script

    Array_Variable[3] AS NUMBER
    Array_Variable[1] = Array_Variable_1_
    Array_Variable[2] = Array_Variable_2_
    Array_Variable[3] = Array_Variable_3_
    OUTPUT_ARRAY_AS_CONSTANTS (“Array_Variable”)

ODBC driver settings
In previous versions of DCS you had to save, close and re-open the run settings to get the correct ODBC driver settings to appear. This bug has been fixed in 9.0 Update 2.

SORT_INPUT
The input data files location in DCS can be set in the run settings or can be set in a particular input format. The parameters to RUN_DCS_PROGRAM by default set the input file location in the input format. In previous versions of DCS, if RUN_DCS_PROGRAM was used and a file location was set in the run settings of the program being called by this function then two input files would be sorted and each input value was used twice. This has been changed in DCS 9.0 Update 2, if the same file is in both the run settings and the input format, the one from the run settings is removed so that the input data is only used once.

9.0.1 DCS Plugins
We have now implemented the Prophet Plugin framework in DCS so that some of the Intex plugin functions can be used in DCS. Only the Intex plugin is currently tagged for use within DCS.

Nested Structures – Grouping from Prophet Professional
One of the enhancements made to Nested Structures for 9.0 Update 1 is the introduction of a new grouping feature. When running a nested structures model, model points are passed from the outer structure to the inner structure through model point maps. Previously you had to pass all model points from the outer to the inner. Now you have the ability execute a DCS script before an inner structure is processed. This script will allow you to perform DCS grouping functionality to reduce the number of model points passed to the inner. Therefore the inner structure will be able to run quicker than before.For more details see the Prophet Professional User Guide.

EBCDIC Data Formats
In previous versions of DCS you could not select the number of decimal places for the following EBCDIC data types

  • Packed
  • Signed Binary
  • IBM Float
  • Binary
  • This has now been addressed so that for the above data types there is a field in which you can specify the correct number of decimal places to use.

Excel Data Inputs
In previous versions of DCS you could only select Excel spreadsheets with an .xls or .xlsx extension. You can now select .xlsb and .xlsm spreadsheets in addition to the standard Excel formats.

There was a bug in DCS 8.2.3 where changes to input formats that used Excel data were not recognised and users were not prompted to save their changes. This bug has been fixed in 9.0 Update 1.

Integer Data Classes
In DCS 9.0 writing Integers or ShortInts to a GDFx file if the data class is Value, not Analysis Factor, was not working correctly; producing random output values.

This has now been resolved and you can correctly use Integer and ShortInt in GDFx output that is a Value data class.

Grouping
Although we made changes to address the issues found with minimum grouping in 8.2. Update 2, there was a remaining issue in 9.0 that meant the run could not continue and write to the output files after the grouping had been completed.

The issue has now been resolved in release 9.0 Update 1.

Input Data Source Names
In previous versions of DCS selecting a specific database as an input would sometimes not be remembered on saving. If you re-opened the Run Settings tab it would have been reset to the default value of [None], this issue has now been fixed.

RE_START Function
There was previously a bug with the RE_START function that could cause a DCS program to crash. This problem has been resolved in 9.0 Update 1.

Tolerance with Comparisons
An error with tolerances when comparing large integer values has been fixed in 9.0 Update 1.

  • One centralized DCS program file to generate model point files for all reporting purposes
    • utilize RUN_PARAMETER
    • Same input format for multiple input (policy data) files
  • Read the table instead of hard-coding
    • Though reading table is slow, the DCS table is usually small
  • Use CODE_SEGMENT to split the code
  • Version control
    • DCS file used to generate model point files for Prophet should be considered as part of the model
    • Proper DCS version control and documentation of DCS changes are necessary
  • Use “rpt” as model point filename extension


A good DCS program will reconcile the data such as number of policies and premiums by producing the following files:

  • Input summary file shows the statistics before any data conversion
  • Skipped records can be due to
    • Lapse or termination
    • Critical errors that affect Prophet results
    • Unmodeled products
  • Output summary shows the statistics of model point files

No matter you use OUTPUT or GROUP to generate model point files, the reconciliation should be consistent. All skipped records should be explained. This follows the audit and control requirements.