It's (ELeM)entary, My Dear Watson

I've used this trick a few times so I thought I should share it on the blog before I get run over by a bus.

Often we (developers) are asked to produce or consume data files for integrations with 3rd party systems.  Even with the emergence of web services there are still many many times where the preferred method of interchange is file based, typically these are bulk (high volume) transactional exchanges.  Think payments between banks etc 

For the systems I've worked on we have solved this with .csv delimited inbound and outbound files.  We have also utilised fixed width files, the majority having old school multi-format layouts with a traditional header, detail and footer structure which has some advantages over .csv.

For now, I will also ignore that we could have also provided XML or JSON payloads as part of a real-time integration. 

As you can see, there are numerous ways to skin this cat. However, this is a 2E and Plex blog so for today, let us concentrate on:-

• How you might build and consume files into a 2E based application. 
• How can we create a multi-format flat file using purely Synon logic only?

Let's fire up a 5250 session and explore an unsung feature in 2E that helps remove the complexity of building these flat files.

What are we going to build? 

An extract file with a header record, 10 detailed transactional records and a footer which denotes both EOF as well as have a total  for basic validation.  In the real-word this may include hash total etc

An example of what this file data might look like is below.

Back in 2E, first define a file with one Known By (Numeric 9.0) should be suffice for most integrations and a second field (Has relation) of 'Flatfile Data' - Type TXT and length of 500 (or whatever length works for your environment). RP4 has better field limits than RPG. 

I've called my file FIXFILP.

Now we need to build the data record, usually you would start concatenating the data values together to meet the design specification of the receiving system and handle those pesky FILLER57 fields etc that always appear from somewhere.

This involves dozens (if not hundreds) of lines of code to concatenate the data.  The resulting action diagram is often difficult to understand and cumbersome to maintain.

What if there was an easier way to build up the data with the ability to easily reorder the fields and cater for changing field lengths.  Well there is, using a neat unsung feature of 2E arrays.

2E keeps track of the last record to be added, changed or read in an array, a sort of cursor I guess.  This is available in action diagram logic hidden in the ELM context. (ELeMent).

The only 'built in' function that can use the ELM context is a *CVTVAR.  

First create an array with the data fields you would like to appear in the dataset, this can be header, detail 1, detail 2, footer etc.  It doesn't really matter for a flat file process.  To keep it nice and simple I have made up some generic fields names with various data types.

I've keyed these arrays based on the Record type of HDR, DTL and FTR.  You can do how best suits your use case.  All the arrays are set with a 'Numer of Elements' of 1 record as I don't need to use the in the traditional sense.  I just need a pointer to an ELM in memory.

All we then do is call a CRTOBJ over the array to populate the data.  Once in the array, we can use the *CVTVAR to populate a flat file field.  2E handles all the different data types and spits out a well formatted string which you can write to the database/extract file etc



But we are not done.  I've ready other blogs that talk about ELM and they do a pretty good job of explaining the outbound example above.  But not many people realise that depending on whether you are using the ELM context as Input or Output, is the equivalent of constructing or deconstructing the data.  So yes, this method can be used to unpack flat files also. :-)

As long as in the receiving function you have created a shell array record.  You can use ELM to move the data into the array and then standard RTVOBJ functionality to retrieve the record in its deconstructed form.

An example below of a couple of screens that I rushed together showing the data string and the subsequent fields.

You simply point the flatfile string to the correct array and 2E will handle everything else.

Thanks for reading.
Lee.

*MOVE and *MOVE ARRAY with 2E date and time fields

Every 2E model has functions that are no longer required, yet they persist and continually get reused.  A good example of this common issue is the continued usage of legacy date conversion routines that exist in most 2E data models.

How many of you have the following functions (or very similar) in your model?

The reason why is quite simple!

 

They were probably written long before 2E supported the date fields indicated.  After all, the DT8 (Date 8) and DT# (Date ISO) were added relatively late in the tools evolution.

I recall working at a company in London (pre 2000), who had solved the Y2K problem by transitioning their date fields from DTE to an user defined field type CDT (Century Date), this was basically a DT8 i.e. YYYYMMDD but implemented way before 2E had supported it.  I believe 2E were quite late to the party and implemented DT8 support around 1999.  Synon Inc started supporting the DT8 field specifically for people transitioning their models from DTE to DT8 and to help overcome any potential Y2K date rollover issues within their code.  There was even a model analysis tool that helped people identify these issues.

The problem is..... Many sites had already solved their issues (as above), whilst others stuck with the DTE format and its limitations and perhaps only targeted certain field like DOB.  I reckon there are dozens of sites around the world that continue to define dates as DTE out of habit!  I would be guilty of doing so for sure.

Anyhow, for those that have moved on and are defining DT8 or DT# or TS# (ISO Timestamp), I still believe that many programmers will be using the self coded legacy functions to perform date conversion.  I know I have too been guilty of this crime from time to time in the heat of coding.

 

The thinking goes something like this?

1. Navigate to your system functions or date functions (scoping file DFN).
2. Filter on Date or Convert (cnv) and pick the function that suites your needs.
3. Test function and everything works fine.
4. Great, Job done!

However, there is one small flaw with this.  You didn't need to use the legacy date conversion functions anymore.  Remember, these were likely written before 2E supported the source or target date format or have been written more recently by a developer who didn't realise that 2E already handles automatic date conversion between its (shipped) data types.

 

As long as you are moving data from a field type that is supported and that it has 'date like' data in it, 2E will automatically handle the conversion for you.

The table below (directly from the 2E online documentation) helps showcase all the automatic conversions that are handled by 2E and below that a table highlighting the limitations or rules.

 

 

* Conversions for the shipped D8# and the user-defined DT8 (8-digit internal representation) data types are identical.

 

What does this look like in the code?  Well let's take a look at some generated code to find out.

The following 'mock up' function is trying to covert the DTE (*JOB DATE) to a DT# (ISO) and DT8 format.

The source (RP4) for this is generated as follows:-

The ISO conversion is a little more complicated and is also generated into a subroutine so that it can be called for any date conversions from DTE to DT#.  The DTE to DT8 conversion is something I am sure you have done many times in code using the *ADD etc these are generated inline and not passed to a subroutine due to the small number of lines of code.

The subroutine code for ISO is below.

A final point to note is that if you have a number field of 7.0 length masquerading as a date, you can move it into a date field DTE date.  Same with 8.0 to DT8, which will be more common if interfacing to data derived from more modern databases that never had a history of supporting dates like DTE or Julian.

You can then apply 2E date functions as usual.

Finally, we are not limited to dates, times and their myriad of formats are also interchangeable.

Thanks for reading. 

Lee.

DSPRCD refresh trick and a few extra tips.

Today I was asked to fix a DSPRCD screen that wasn’t refreshing after an up-screen EDIT function (Command Key) had changed some values. Upon returning to the DSPRCD it still showed the original values.

DISCLAIMER: This is correct behaviour for a 2E function but a little bewildering for the user as they are hesitant as to whether they actually made the change in the update screen.

For a DSPFIL we would use the *reload subfile to force a reload of the screen.

For a PMTRCD we can set the function option ‘Repeat Prompt’ to get the same behaviour.

For DSPRCD etc it is a little trickier…….

Typically, I have seen people utilise a driver program and execute the screen in a loop and simply recall the screen or exit.  This is doable……    There is however, (as always)…. An alternative, a trick and as it is much simpler I will show you.

The answer is to force the transaction to continue.  You maybe be aware of the PGM.*Continue Transaction field in 2E.   It is typically used (in action diagrams) for the DSPTRN and EDTTRN function types.  The same field is used in many AD’s including the DSPRCD.

In this instance it is simply there to initiate a loop.  Typically this field only has one condition (*NO) for using the EDTTRN and DSPTRN.  See these notes in the 2E manual….

https://techdocs.broadcom.com/content/broadcom/techdocs/us/en/ca-enterprise-software/it-operations-management/ca-2e/8-7/building/building-applications/modifying-action-diagrams/understanding-action-diagram-user-points.html

The trick is that the DSPRCD generates code that says if W0TRN (*Continue Transaction) = ‘R’ then keep re-showing screen. 

Our issue (default model)  is that we have no way (default way) of setting it…… We do now. 

I have simply added a new condition to the field *Continue Transaction (I’ve done it at many sites).  The condition is called ‘*Reload Trick’ and we just need to set in after we’ve called our screen.

Please note you may need to follow this with a *QUIT depending on your circumstances.

Voila…. It works 😊

Additionally, did you know.....?

1. USER:Process Command Keys has got nothing to do with command keys. Command key processing should normally be added to USER:Validate Detail Screen. 

or for DSPRCDx

1. The correct screen context DTL/2ND/3RD must be used for error message parameters to ensure that field is highlighted. Normally in any action diagram coding the screen context is not significant.
2. The validation cycle processes data from all pages together, therefore, the relation settings apply to the function as a whole, and not to individual screens.
3. It is not possible to control, which page the user sees. If there is an error the function always displays the first page, which has an error outstanding, but this may not correspond to the actual error message displayed on the message subfile line. The messages are displayed according to the sequence they were sent. Therefore, you may have to review the validation sequence.

Tip: Validate a page at a time and only start validating a subsequent page if *PGMERR is not set.

Thanks for reading. 
Lee.

A little trick with *SET CURSOR

A quick little post to kick off 2019....

A colleague of mine had an issue today where he was trying to stop a user having to page down dozens of pages when inserting data via a DSPFIL/PMTRCD W/W (Work with) suite.

He asked me how he can reload the subfile (show the new data) but position the page at the point he (the user) was at, rather than refreshing and defaulting back to the first page again....


The solution is quite simple and as the title of the blog says.......

You use the *SET CURSOR function and set it to a field on the subfile record you wish to remain in focus.  In our case we chose the *SFLSEL field.



Thanks for reading.
Lee.

Multi-Line Edit Oddities

Hiya,

Today I helped a colleague (a very talented one at that) with a small issue around a PMTRCD and usage of multi-line edit for oversized fields.  Something he hadn't seen before.

He was confused as to why his data entry field was showing on the device design and not showing when executed.

Figure 1 shows a mock up of his device design looking pretty standard.

Figure 2 shows the same screen at run-time.  Note the cursor has positioned to the field acting strangely but the underline is not showing.

If you want to recreate this issue I created a file as follows (See Figure 3 and 4) :-

 

Additionally, in the device design we set the long field to multi-line Y and set it to 4 rows and 50 wide.

To resolve the issue was a case of applying a (clearly little known) trick of moving the multi-line field over by one byte.  (Figure 5) 

 

This then allows the run-time screen to paint properly. (Figure 6)

My assumption is that the screen attributes are being corrupted by the multi-line although comparing the source for the DDS didn't highlight anything untoward so I am guessing it is a 5250 issue.  (I also recommend aligning the rest of the fields on the screen.)

Any takers on providing the technical explanation? as a compare source doens't seem to highlight a clear and present danger. (Figure 7)

I've always nudged these over by an extra byte for years and years but I guess some tips and tricks get lost.

Thanks for reading.
Lee.

Don’t forget the *return code

When we are developing in 2E we often reach the 50 file limit if we are generating for RPG.  RPG ILE (RP4) is more resilient as it allows more files to be opened.  I’ve preached before regarding proper function construction. 

Do you really need 50 or more files open for any given process? 

Can your function be better constructed or to be accurate………deconstructed?

What are the options?

1.       The easiest method to get over these limits is to change to RP4.

2.       The best is probably to re-architect your function properly and switch to RP4.

3.       The next best approach is to probably hive off a chunk of the processing into a new function to reduce the open files.

4.       The worst thing to do, is to externalise a single RTV in order to get under the limit.  As this can cause all sort of issues.

o   The 50 file limit will be breached on the next maintenance (most likely) leaving the potential for a string of EXT/RTV type functions.

o   If called in a loop or RTVOBJ then an external (if not set to Close Down ‘No’) may impact performance.

Another item to add the list of don’ts (above) is the preservation of the *Return Code.  An Execute External Function does not automatically return a *Return Code.  By default, standard processing is to pass back *Normal or an empty *Return Code.

If you are relying on a *Return Code for your now externalised RTVOBJ it will always come back as empty (*Normal)

unless you explicitly pass the return code back with the *EXIT PROGRAM built in function.

 

Another trap for young players but something that catches even seasoned developers out.  Had the RP4, re-architect or block of code to EEF option been taken the issue was likely to not have manifested.  

Good programming practice is to always be conscious of the *return code and more importantly (where required) test for it.

 
Thanks for reading.
Lee.

Under the covers of the SELRCD function type

Hi,

This blog touches on the design challenges when using select record (SELRCD) function type and makes some recommendations for best practices.  Disclaimer:  These are from personal experience and may not cover all scenarios.  I would be happy to hear of any ideas/improvements that can be made to this post.

SELRCDs can cause quite a few issues if you are not aware of the quirks of how Synon generates the code.

Let's cut straight to the problem domain at hand.  In my sample model you will see that I have two files declared.  You will see that "Lee's File" Refer to "Lee's Reference File" which will mean that if I have a PMTRCD or EDTFIL etc over Lee's file we will be able to prompt for the reference data.

The functions I have over the reference file are as follows, I have removed the default EDTFIL and renamed the database primitive functions inline with our internal development standards.

 The main file has numerous test functions so I have filtered on Edit file only.

The screens (Edit File and Select Records) in the blog post have been tidied (ever so slightly) to ensure that all the fields fit nicely on the screen.

I have generated up the code for the EDTFIL.

Reviewing the generated code.

By default you will see that Synon has generated code to call the SELRCD function upon prompting the field and Synon has chosen my SELRCD based on the reference file.

You will also notice that the parameters for the SELRCD have been defaulted to fields based on the RCD context of the calling program.  In this instance it was an EDTFIL.

This is pretty standard and works a treat.

Now we will add a few additional SELRCDs, you will see these below.  Note the naming of these was deliberate in order to ensure that they appeared after 'Select Lee's Reference Fi'.

In terms of impact analysis you will notice that the SELRCD is showing no usages although we have generated code above that proves the object is called.

This is due to implicit code generation that is automatically added if the field is a result of a foreign key constraint AND is input capable AND a select record exists on the reference file.  If you deleted the SELRCD at this stage and generated the code again the prompt and call logic will not be generated.  Give it a try!

 

Above is a screen image of the default Synon setup.  Notice the two subfile options at the bottom of the screen.  S allows you to select the SELRCD that should be used, T clears this value and leaves the generator to choose the default SELRCD.

After taking the S option and selecting my SELRCD from the reference file you will see that this is indicated above.  Because we have made an explicit reference to the SELRCD in the device design it will now appear as a usage within Synon.

You will see below that the generated code is as before which is great :-)

In order to help with the blog narrative, please now deselect the SELRCD by taking the subfile option T for default behaviour.   This will clear the reference to the dedicated SELRCD.

Now we will create another SELRCD (again deliberately named).  You will see that it sits above the current (default) select record for the EDTFIL.

If we generate the EDTFIL again we will see that the outcome is no longer the same. The generator now puts a call into the higher ordered SELRCD on the reference file.  We didn't intend for this to happen and a developer doing a regeneration could easily pick this up without knowing.  Quite dangerous....

In this instance, as the keys on the SELRCD are identical there is NOT too much to worry about.  However, in the real world 2nd and subsequent SELRCD functions will typically be over an alternate access path with a different key structure or may have additional parameters or specialist action diagram functionality which may not be the preferred default behaviour.

In order to demonstrate this let's add some fields to the model which we will then associate with a new SELRCD function.

Create yet another SELRCD function over the referenced file.  Note again I have named it deliberately so it appears at the top of a refreshed EDIT FUNCTIONS panel.

I have extended the parameters for this SELRCD to add our five new fields.

Let's generate up the EDTFIL once again and look at the generated code.

Ouch!!  Houston we have a problem.

You will see below that all of the new parameters have been defaulted to CON.  This is because the fields were not found in context for the relationship. i.e. in this case, not part of the RCD context. And because this is a warning message only it may go unnoticed.  Preferably Synon should mark these as errors and only allow the correct context ones to generate correctly i.e RCD example above.

Also we have now picked up yet another SELRCD.  One can conclude that anytime a new SELRCD is added and it appears higher up in the EDIT FUNCTION panel the next time a referencing function is generated it will generate code to call the new SELRCD and not the previous one.  Imagine a highly reference file and the chaos this could bring.

We can do somehing to help here.  Whilst we cannot get at the parameters at the point the implicit call to the SELRCD is made, we can put these fields in the context and populate them accordingly.

Add the fields to the 'Subfile Record' format RCD.  You will notice I set these to hidden.

Then in the action diagram you can set these values using a standard *MOVE built-in function.

Generate the function once more and the generated code now shows the moved and also defaults to the RCD context meaning that you no the parameters are populated with your values and NOT CON blank or Zero.

But I remind you at this stage that these were warnings in the code only.  How many people check the generated source for warnings.  I don't, do you?

Another way to do this is to switch off default processing and take control of the prompting yourself in the action diagram coding.  Back in the EDTFIL function again set the relation checking to 'USER'.

Note: This switches off all referential integrity checking so in the real world you'll also need to add action diagram code for the OPTIONAL/MANDATORY as well as foreign key constraints.  For the purposes of this blog I am only focused on the prompting and calling of the select record.

You can then monitor for prompting on your field and call ANY SELRCD function you think is appropriate.

With this method you get a nice parameter interface for the SELRCD and more importantly it is obvious to another developer that you are initialising the fields for an explicitly called SELRCD rather than an implicitly called on.  Some comments might be useful if you choose the other method.

This highlights the many options you have for calling a SELRCD and should help you decide what is best for your model, team etc.

Once last issue with SELRCDs.  I mentioned earlier that Synon (by default) will chose the highest SELRCD based on the referenced file.  Due to no usages showing it is possible that code will be generated to call an object that hasn't been created or has been removed by a developer thinking it is not used.  This will mean you'll get a 'resolve to object' call error which isn't a good look.

Best tip is to generate up a standard SELRCD for each reference file and implement it at the time the file is created.  If you decide that a file doesn't need prompting, then remove ALL SELRCDs from the file so this code doesn't get generate implicitly.  As you can see from above the code will get generated if the function exists regardless of whether the object exists or not.

Best Practices Summary

1. Come up with a naming convention for the default SELRCD so that it is first in the list on the EDIT FUNCTIONS panel.  We use '*SLT xxxxxxxxxxxxxxx'.
2. Always ensure you generate up and implement ALL SELRCD functions.
3. Always keep the default SELRCD keys the same at the RTV access path.
4. Always name 2nd and subsequent SELRCD so they appear lower in the EDIT FUNCTIONS panel.
5. It is recommended you set explicitly the SELRCD you want to call either by the 'S' option or via action diagram coding.
6. Never delete any SELRCD objects without checking the generated source files or by exploring all file references.

Thanks for reading.
Lee.