Data Conversion System (DCS)

• Improved copy-paste functionality in the run settings
• Support for Microsoft Access Database Engine 2016 redistributable
• Improvement to performance of table reads

• New licensing solution

• New functions added
• Generic table functions now support absolute file paths
• Options dialog improved
• Debugger enhancements

• OUTPUT_ARRAY_AS_CONSTANTS enhancements
• Significant increase to DCS limits
• Improved table creation

• Copying input and output formats
• Nested structures grouping improvements

• DCS Plugins
• Nested Structures – Grouping from Prophet Professional

• DCS Plugins
• Nested Structures – Grouping from Prophet Professional

• Improved rounding tolerances
• New functions

• Improved debugger, including conditional breakpoints
• Improved saving functionality

• An issue where mismatching capitalisation of text variables in the Code and Output format would result in Grouping errors has been fixed.
• An issue with locking of Access Database output files has been fixed.

Support for Microsoft Access Database Engine 2016 redistributable
We now support the use of the Microsoft Access Database Engine 2016 in DCS, so that Microsoft Access 2016 and Microsoft Excel 2016 can be used as input and output data sources, with the latest performance enhancements and bug fixes.

Improvement to performance of table reads
The underlying code for READ_GENERIC_TABLE statements has been modified so that performance of DCS scripts with large numbers of such calls can see improvements. Note that performance will vary across workspaces and is dependent on a number of factors including hardware and compiler used.

• An issue which prevented scripts from outputting to both GDFx and ODBC file formats in the same run has been fixed.
• The compilation log sometimes included a lot of white space in Windows 10. This has now been addressed and no white space appears.
• An issue where the NUMVAL function incorrectly reported an overflow/underflow error has been fixed.
• Boundary validation on integers and short integers in the input format has been corrected to catch overflow and underflow errors.
• DCS now correctly reports all tables used within the script.

• User licences: Tied to an individual user’s Active Directory account such that only the Prophet user with the email specified on the licence can acquire it. This is similar to the previous Machine-locked solution.
• Group licences: These are licences that can be used by multiple users (from a defined list) within an Active Directory group, as controlled by their email addresses and selected by the customer. This is similar to the previous Floating licence.
• Site licences: These are licences that can be obtained by anyone who has access to the client licence server that is distributing the licence. The technology is safe for virtualised environments.

New licences will be allocated as follows. Machine-locked licences will be transferred, one-to-one, into User licences. Floating licences will be transferred, one-to-one, into Group licences.

New licences will be pushed out by us to your Client Licence Server (CLS). The CLS synchronises with the FIS Licence Server (FLS). The CLS therefore requires access to the internet and individual machines require access to the CLS. If a disaster event occurs and the CLS is lost then a new CLS simply connects to the FLS where replacement licences are synced.

If you require more advice or support regarding the changes to the licence solution please contact FIS Client Services or your local FIS representative.

Installation location
Prophet installed files are now placed in C:\Program Files (x86)\ProphetSuite, C:\Program Files\ProphetSuite and C:\ProgramData\ProphetSuite or equivalent.

• Some issues around how plugins interact with the runlog and DCS script have been fixed.
• The debugger would become unresponsive when trying to debug an EBCDIC workspace. This has been fixed.
• Copying a workspace that uses a DRP file could cause the run to fail as it would still try to use the original location. DCS has been changed to use the DRP file in the location of the runtime executable.
• DCS would crash if input fields had more than 36 characters. This has been fixed.
• When using the 64 bit 2015 compiler the debugger may crash whilst running. This has been addressed.
• An issue when that caused the DCS run to fail using 64 bit with Excel output has been fixed.
• Issues with additional format records and ODBC input caused double records to be outputted. This has been fixed.

User interface
Improvements and fixes to the user interface include:

• Attempting to expand the program node by double clicking causes DCS to try to open it as a table in Prophet Viewer.
• An exception was previously given when using DCS Compare if the user tries to go to a line beyond the end of the file. This has been fixed.
• DCS would previously crash when attempting to use the Insert Variable menu. This has been fixed.
• When using an xls file as an input DCS would fail to open the input record when prompted. This has been addressed.
• An error produced when creating a new input format for a new DCS workspace has been fixed.
• Scrolling in the output format would cause the variables to be grouped to be checked. This has been fixed.

• GET_EXECUTABLE_LOCATION: Returns a text value containing the full path of the current DCS program executable.
• GET_CURRENT_OUTPUT_FILE_PATH: Returns a text value containing the full path of the output file location.

The following functions now support absolute table paths as well as relative:

• GENERIC_TABLE_CLOSE
• GENERIC_TABLE_EXISTS
• READ_GENERIC_TABLE
• READ_GENERIC_TABLE_TEXT

User Interface
The settings in the Options dialog box have been grouped by Workspace Options, User Options and Machine Options to make it more consistent with other Prophet applications.

The DCS program name has been added to the title bar and taskbar so that you can distinguish between programs easier.

Input and output formats now appear in alphabetical order with sub-records and additional records as child nodes of their parent main format.

Debugger
The DCS debugger has been improved to make it generally easier to use:

• Watches and panel state remembered on exiting and re-entering debug mode.
• Breakpoints automatically adjust when new lines of code are added in the code editor.

• An issue which meant that a DCS run would fail if a dynamic array was set as a sorting/grouping variable in an output format and passed into an OUTPUT_ARRAY_AS_CONSTANTS call has been resolved. This means that dynamic arrays which are text variables can now be included in the output format.
• We have fixed an issue that caused scripts using arrays of size greater than 32768 to fail at runtime.
• An issue where RE_START was failing with additional record input formats has been fixed.
• The DELETE_EXISTING_OUTPUT_FILES function would not work if the program had no input files, this has been fixed.
• Using NEXT_RECORD as part of a procedure would move out of that procedure when processing the current record rather than moving onto the next one. This has been fixed so that NEXT_RECORD now works correctly in procedures.
• We have fixed an issue where READ_GENERIC_TABLE was not correctly respecting the “Maximum number of runtime errors before termination” value set in the Run Settings.
• Some issues with ODBC output to multiple tables have been fixed.
• Array out-of-bound resizing has been made more performant by using a geometric rather than arithmetic expansion algorithm.

User Interface
Improvements and fixes to the user interface include:

• Auto-refresh of various objects (such as the results tree) following a run.
• Debugger fixes:
  • Watches pane shows correct values for multi-dimensional arrays.
  • Panel resizing now correctly persisted on closing and reopening.
  • Programs with modules now run without issue in the debugger.
• Modules improvements:
  • The user interface was found to have issues handling modules with long file names; file names have now been restricted to 64 characters to prevent these occurring.
  • Modules can now be opened without error from Windows Explorer.
• Categorisation of functions in the Function Wizard has been improved and a few cases where an incorrect function description was being displayed have been resolved.
• The compile dialog is now modal, disabling the main user interface and preventing changes to compile-time dependent program settings.
• The compile dialog edit button, available when selecting compile errors at the end of a failed compile or when viewing the compile log, now consistently opens the code editor to the line number shown in the error.
• The User Interface is now partially and consistently locked down during a DCS run to ensure that actions that would attempt to delete the running executable are not permitted.
• Case-insensitive check performed on creation of input and output formats to prevent duplicates being created.
• We have fixed an issue where clicking ‘View Formatted’ with an Excel input format open would cause DCS to error.
• Rename Variable functionality has been improved to prevent duplicate variables being declared or unintentional merging of two variables into one.

Increase to the number of input/output fields supported
DCS has been extended to support the import of formats with more than 255 input and output fields. The limit will depend on the data format that is used. Please note that viewing a file in DCS with a large number of fields may cause the user interface to perform slowly.

Increased text variable size limit
The maximum text variable size has been increased from 65 to 255 characters to make it consistent with Prophet.

DCS output metadata
DCS can be used to create tables for Prophet. We have improved this functionality so that tables and model point files produced by DCS will now contain the metadata that allows Prophet to recognise the type. The file will now open in Prophet without requesting the file type and will be shown in the appropriate section of the user interface.

ODBC
We have fixed some issues that prevented DCS from using some database types via ODBC.

Fixed Size Arrays
Some further checks have been applied for fixed size arrays. This may cause some DCS scripts to start failing where they had previously succeeded. An appropriate error is given if the variable is assigned a value for an element above the declared size.Fixing the DCS script in such cases can be straight forward for users as the array can be declared with 0 size. This effectively makes the variable a dynamic sized array which can grow as required.

Nested structures grouping improvements
The functionality to call DCS from Prophet during a nested structures run has been improved in DCS 9.0 Update 2, specifically we have made updates to

• Grouping in models that use new business
  In the previous version of DCS, if the Prophet model contained new business model points then the DCS grouping functionality would not work because of the order of processing new business in Prophet. DCS and Prophet 9.0 Update 2 have been enhanced so that the existing business model points are passed to DCS and grouped. New business model points are appended to the grouped existing business and are then passed to the inner structure to be processed.
• Population of arrays in the DCS script
  In previous versions of DCS array variables would be populated in DCS input formats with brackets around the array index, for example,

  Array_Variable(1)

  Users had to manually convert the brackets into underscores to save the input format. DCS population of input formats has been improved so that the conversion of array index brackets to underscore characters is done automatically.

  For multi-dimensional arrays the commas separating the indices are also converted into underscores, for example,

  Array_Variable(1,1,3)

  would be populated in the input format as

  Array_Variable_1_1_3_

  ---

  Note: This has been done for nested structures grouping input formats as well as all standard DCS input formats.

  ---

  In addition to the input being auto-populated for array variables DCS grouping scripts are also auto-populated with code to convert the inputs into a DCS array and use the DCS OUTPUT_ARRAY_AS_CONSTANTS formula to output the array back to the Prophet model, for example if you had an array in your model point file with 3 indices then the following code would be automatically populated in the Code of your DCS grouping script

  Array_Variable[3] AS NUMBER
  Array_Variable[1] = Array_Variable_1_
  Array_Variable[2] = Array_Variable_2_
  Array_Variable[3] = Array_Variable_3_
  OUTPUT_ARRAY_AS_CONSTANTS (“Array_Variable”)

SORT_INPUT
The input data files location in DCS can be set in the run settings or can be set in a particular input format. The parameters to RUN_DCS_PROGRAM by default set the input file location in the input format. In previous versions of DCS, if RUN_DCS_PROGRAM was used and a file location was set in the run settings of the program being called by this function then two input files would be sorted and each input value was used twice. This has been changed in DCS 9.0 Update 2, if the same file is in both the run settings and the input format, the one from the run settings is removed so that the input data is only used once.

Nested Structures – Grouping from Prophet Professional
One of the enhancements made to Nested Structures for 9.0 Update 1 is the introduction of a new grouping feature. When running a nested structures model, model points are passed from the outer structure to the inner structure through model point maps. Previously you had to pass all model points from the outer to the inner. Now you have the ability execute a DCS script before an inner structure is processed. This script will allow you to perform DCS grouping functionality to reduce the number of model points passed to the inner. Therefore the inner structure will be able to run quicker than before. For more details see the Prophet Professional User Guide.

• Packed
• Signed Binary
• IBM Float
• Binary
• This has now been addressed so that for the above data types there is a field in which you can specify the correct number of decimal places to use.

Excel Data Inputs
In previous versions of DCS you could only select Excel spreadsheets with an .xls or .xlsx extension. You can now select .xlsb and .xlsm spreadsheets in addition to the standard Excel formats.

There was a bug in DCS 8.2.3 where changes to input formats that used Excel data were not recognised and users were not prompted to save their changes. This bug has been fixed in 9.0 Update 1.

Integer Data Classes
In DCS 9.0 writing Integers or ShortInts to a GDFx file if the data class is Value, not Analysis Factor, was not working correctly; producing random output values.

This has now been resolved and you can correctly use Integer and ShortInt in GDFx output that is a Value data class.

Grouping
Although we made changes to address the issues found with minimum grouping in 8.2. Update 2, there was a remaining issue in 9.0 that meant the run could not continue and write to the output files after the grouping had been completed.

The issue has now been resolved in release 9.0 Update 1.

Input Data Source Names
In previous versions of DCS selecting a specific database as an input would sometimes not be remembered on saving. If you re-opened the Run Settings tab it would have been reset to the default value of [None], this issue has now been fixed.

RE_START Function
There was previously a bug with the RE_START function that could cause a DCS program to crash. This problem has been resolved in 9.0 Update 1.

Tolerance with Comparisons
An error with tolerances when comparing large integer values has been fixed in 9.0 Update 1.