pasUnity 4.42.0.0 RTM is now generally available.

This release adds a new tracking tables feature that allows workflows to persist values between executions..

Sometimes you have to persist a value between executions of a job.  As pasUnity is infinitely extendable you could have created your own tracking mechanism using a variety of different technologies but why should you have to?  We now have a very efficient, lightweight, easy-to-use solution to this problem as introduced in this release as the Key Value Pair management system.  This new feature includes the ability to store character values of any size associated with either the system as whole, an agent, a job, or even a specific impeller using a key unique to the scope at which it is defined. 

To manipulate these key/value pairs you then use the <KeyValuePair> macro command which allows you to Get, Set, or Increment values.  The syntax for this command is: <KeyValuePair|action|scope|key|value> where action is either get/set/increment and scope is either System/Agent/Job/Impeller, and the key and value (optional) are user supplied.

Scenario 1: Automatically incrementing number in a file name.

Scenario:  Every time a specific integration job runs it needs to create an output file named DATAFILEX.TXT where X is an auto incrementing number each time you save the file. 

Problem: Up until now if you wanted this capability you would have to add procedural code job steps to you job (such as the SQL Query, Managed Code, or PowerShell Script job steps) to your job that at the beginning of the job would go read a value from a table, possibly modify that job, do some work, and at the end write that value back to a database or file.  These tasks that are moderately easy to implement if you have some programming knowledge, but even still you might run into problems because a job failed or maybe you did not take concurrency into consideration when designing your tracking tables.

Solution: Use the new <KeyValuePair> macro to create a parameter on the job itself to retrieve the next sequential number and generate a file name all once.  

Example:  Parameter TargetFileName with value DATAFILE<KeyValuePair|Increment|Job|FileName|1>.txt all will generate a filename of DATAFILE1.txt the first time and DATAFILE2.txt the next time it is run.  There is no need to pre-stage the key/value pair so that macros could be used for the key as well as the value.

Scenario 2: Automatically incrementing number in a file name across multiple jobs.

Scenario:  Every time any one of several integration jobs run it needs to create an output file named DATAFILEX.TXT where X is an auto incrementing number each time you save the file and never reuse the number across any of the jobs.

Problem: Same problem as scenario 1 but at a higher scope: the agent instead of the job.

Solution: Use the new <KeyValuePair> macro to create a parameter on the job itself to retrieve the next sequential number and generate a file name all once.  

Example:  Parameter TargetFileName with value DATAFILE<KeyValuePair|Increment|Agent|FileName|1>.txt all will generate a filename of DATAFILE1.txt the first time and DATAFILE2.txt the next time it is run.  By specifying System you could even have made the name unique on multiple agents.

Scenario 3: Use Get and Set to persist a non-incrementing value.

Scenario:  A singles job with multiple impellers each for a different business unit that extracts all ledger activity since the last successful execution.

Problem: Each impeller needs to track the success date independent of the other impellers to ensure that data does not get pulled multiple times and that data does not get missed.

Solution: Use the new <KeyValuePair> macro to create a parameter on the job itself to retrieve the last success date and another parameter to for an effective stop date.  Additionally, add query logic to filter on the new parameters.  Finally add a parameter job step that writes the new start date back to the database on success.

Example:  Parameter StartDate with value <KeyValuePair|Get|Impeller|StartDate|<DateTime>> will give you the start date.  Parameter StopDate with value <DateTime> will give you the stop date.  Then implement a query with logic similar to SELECT * FROM Ledger WHERE TransactionDate BETWEEN '<Parameter|StartDate>' and '<Parameter|StopDate>' to get the data.  Finally, add a parameter job step to the end of the job to write the value back to the database like this: <KeyValuePair|Set|Impeller|StartDate|<Parameter|StopDate>> 

On top of the new macro there is also a set of reports that allow you to see where Key/Value pairs are in use, edit values, pre-stage values, and more.  To begin the System, Agent, Job, and Impeller editors have been enhanced with a Key Value Pairs tab as shown below:



This view enables all sort of new scenarios.  For example, in testing you watched as the Increment action started as 1 and worked itself up to 20 before you were satisfied with the job.  Well, now you can use the editor to set the value to 0 so that next time it generates a value of 1 once you put the job into production.  Or you could just delete the line and let the macro recreate it.

Pretty cool right?