Showing posts with label Arrays. Show all posts
Showing posts with label Arrays. Show all posts

Saturday, December 2, 2023

It's (ELeM)entary, My Dear Watson

I've used this trick a few times so I thought I should share it on the blog before I get run over by a bus.

Often we (developers) are asked to produce or consume data files for integrations with 3rd party systems.  Even with the emergence of web services there are still many many times where the preferred method of interchange is file based, typically these are bulk (high volume) transactional exchanges.  Think payments between banks etc 

For the systems I've worked on we have solved this with .csv delimited inbound and outbound files.  We have also utilised fixed width files, the majority having old school multi-format layouts with a traditional header, detail and footer structure which has some advantages over .csv.

For now, I will also ignore that we could have also provided XML or JSON payloads as part of a real-time integration. 

As you can see, there are numerous ways to skin this cat. However, this is a 2E and Plex blog so for today, let us concentrate on:-

  • How you might build and consume files into a 2E based application. 
  • How can we create a multi-format flat file using purely Synon logic only?

Let's fire up a 5250 session and explore an unsung feature in 2E that helps remove the complexity of building these flat files.

What are we going to build? 


An extract file with a header record, 10 detailed transactional records and a footer which denotes both EOF as well as have a total  for basic validation.  In the real-word this may include hash total etc

An example of what this file data might look like is below.


Back in 2E, f
irst define a file with one Known By (Numeric 9.0) should be suffice for most integrations and a second field (Has relation) of 'Flatfile Data' - Type TXT and length of 500 (or whatever length works for your environment). RP4 has better field limits than RPG. 

I've called my file FIXFILP.




Now we need to build the data record, usually you would start concatenating the data values together to meet the design specification of the receiving system and handle those pesky FILLER57 fields etc that always appear from somewhere.

This involves dozens (if not hundreds) of lines of code to concatenate the data.  The resulting action diagram is often difficult to understand and cumbersome to maintain.

What if there was an easier way to build up the data with the ability to easily reorder the fields and cater for changing field lengths.  Well there is, using a neat unsung feature of 2E arrays.

2E keeps track of the last record to be added, changed or read in an array, a sort of cursor I guess.  This is available in action diagram logic hidden in the ELM context. (ELeMent).

The only 'built in' function that can use the ELM context is a *CVTVAR.  

First create an array with the data fields you would like to appear in the dataset, this can be header, detail 1, detail 2, footer etc.  It doesn't really matter for a flat file process.  To keep it nice and simple I have made up some generic fields names with various data types.







I've keyed these arrays based on the Record type of HDR, DTL and FTR.  You can do how best suits your use case.  All the arrays are set with a 'Numer of Elements' of 1 record as I don't need to use the in the traditional sense.  I just need a pointer to an ELM in memory.



All we then do is call a CRTOBJ over the array to populate the data.  Once in the array, we can use the *CVTVAR to populate a flat file field.  2E handles all the different data types and spits out a well formatted string which you can write to the database/extract file etc



But we are not done.  I've ready other blogs that talk about ELM and they do a pretty good job of explaining the outbound example above.  But not many people realise that depending on whether you are using the ELM context as Input or Output, is the equivalent of constructing or deconstructing the data.  So yes, this method can be used to unpack flat files also. :-)

As long as in the receiving function you have created a shell array record.  You can use ELM to move the data into the array and then standard RTVOBJ functionality to retrieve the record in its deconstructed form.




An example below of a couple of screens that I rushed together showing the data string and the subsequent fields.



You simply point the flatfile string to the correct array and 2E will handle everything else.

Thanks for reading.
Lee.


Thursday, February 9, 2017

2E Code Review - pet hates - part I


Today I thought I'd get a few things off my chest. 

Whenever I review code, look at old code or have the job of maintaining code (not my favourite part of the job by a long shot) I am often left dumbfounded.  I see the same old mistakes being made regardless of where I have worked. You will often hear me saying,"Whoever wrote this should be shot!!".

The most annoying part is that in many places the developers agree about the best way to code but quickly use the excuse of "I don't have enough time", or "Well it's done now, so don't worry" to not code properly.  My experience tells me it is often the most experienced developer(s) who are the hardest to convince of solid development standards with the terms "Old dog" and "new tricks" preeminent in my mind....

IMHO the difference between a run of the mill programmer (I've worked with plenty of these) and a good one (fewer but they are out there) is adherence to the little details that make long term maintainability of your code as easy as it should.  This is especially true considering the inevitable change and enhancement that is required during an applications growth and evolution.  I have espoused the standard development quote that 90% of an applications life cycle is maintenance numerous times before.

Anyhow, my post today is a list of pet hates that I see when people are "unprofessional", "selfish","crude", "Lazy" with their 2E coding.

1. Legacy commented out code.

I recently worked on some maintenance and had to work through some action diagram code.  Using the Find Services option within 2E I started to quickly get frustrated that the majority of the usages that were commented out.

What was worse still is that there were developer comments (from 2002) stating that the code was commented out and I know that this area has been enhanced many times since.  There is simply no reason to keep such old code.  The developer concerned is normally quite good but has a couple of really bad habits like this one.

Here is a snippet of code from an old function (screen print kindly shown with permission).  Note, I have redacted any client or developer or brand comments (hence some empty white space).

 

I am all for commenting out code as part of debugging, rapid prototyping, unit testing and quick wins (hot fixes) where you are not quite sure of the change you are making, but please annotate when and why.  However, I am aware that some of this code was moved to their own program(s) so IMHO should just have been removed.

Associated hates:
  • Having to weed through the chaff to get to the code to change.
  • Any old code will not have been maintained so will not (often) be fit for purpose (if you decided to uncomment it) so why leave it there.
  • Extra impact analysis (especially for internal functions). 
  • Confusion with the impact analysis usage.  A future post is planned for this.
  • Better to take a version.
  • Commenting out code doesn't change the timestamp for the AD line so we don't know when it was commented out.
Tip: Comment out code sparingly and preferably not at all.  Be confident with your code and solution.  After you have completed the work (and tested it), if you are 100% happy with the results revisit the action diagram and remove commented out code.  Commented out code is a maintenance burden and others will not understand why.

Here is the same code with the bad commented out code and legacy comments removed.

Still not perfectly structured but a whole lot more readable.  Perhaps CA can add a hide/show commented out code option for the action diagrammer.

2. "GET All fields" RTVOBJ not doing as described.

Every 2E developer has written the standard RTVOBJ that returns the full record. Yes, we all have forgotten the *MOVE ALL, but my biggest annoyance is when people don't visit these functions when the underlying file structure changes.

Imagine the "*GET", "GET Record", "Get All", "RTV All Fields" function without the new fields as parameter.


The next developer comes along and low and behold, there are a few fields missing.  Then they create a new one called Get All (New) and probably flag the other one with some kind of encoding system like DNU for Do Not Use etc.

Associated hates:

Numerous versions of the same type of function.  Too many choices.
Description vs reality can be misleading and frustrating.

Tip: If you have to change the file, you may as well change the "Full Record" type function as you have to regenerate up everything anyhow.

3. Using all 9 parameter definition lines for functions parameters.

In the old days when we only had *FIELD, Access Path(s) or Structure files for defining parameters it could sometimes get a little busy and we'd fill the 9 lines leaving limited options for the next developer.  We shouldn't create structure files just for the sake of it, after all, their primary purposes was for consistency of repeating data groups like audit stamps etc not for easier passing of parameters.

Since version 4.0 (over 20 years) we've been able to define parameter arrays.  There really is no excuse now for taking up the last line of a functions parameter block unless "I don't have time" or "I'll do it when we get to 10" is a valid reason.

Tip: When maintaining a function and say 6 or more parameter block lines are used consider refactoring with a parameter array.

I still think that even using parameter arrays as being a bastardised solution for this problem and that 2E should just have had more that 9 lines....but it is the lesser or two evils.

This is the tip of the iceberg.  Plenty more to follow I am sure.

Thanks for reading.
Lee.

Wednesday, January 20, 2010

Implementing a 'Generic' Data Driver File + Printing/Displaying Arrays in Subfiles (Part III)

Firstly.  Sorry for the delay in finishing off this series.  I have been away on holiday for 6 weeks and when I returned I have been in hospital having my knee operation.  I am now in recovery (bed rest) and finally have a little spare time to finish off the blog.

So let's do a little recap. 

In part one, I discussed the merits of a generic data driver file.  This is a different approach to normal 2E data driven programming but as indicated is particularly useful for implementing *Arrays for DSPFIL's or a PRTFIL or for merging head/footer details into one DSPFIL/PRTFIL.

http://leedare-plex2e.blogspot.com/2009/11/implementing-generic-data-driver-file.html

In part two, I was merely trying to walk you through the solution I was required to provide for one of my customers.  These screenshots have been modified from their original form for confidentiality reasons but are posted with permission from my employer http://www.sasit.co.nz/.

http://leedare-plex2e.blogspot.com/2009/11/implementing-generic-data-driver-file_20.html

Today I am just going to do a quick walk through of what was required to create the solution.  With a little bit of effort (and luck) you should be in a position to work this out for yourself.  Of course, I am always happy to take questions and assist if neccessary.

Step 1.  Implement a Data Driver file.

A very simple file with one key.  I just made mine a simple numeric field.



Here are the field details




I just went with the default sizes for a NBR field.

Remember, once you have created this file you will need to populate it.  I simply populate this with two records 1 and 2 as the keys.  I have seen other implementations where people have put all 99,999 records in the file to make their programming a little easier.  I prefer a slightly different method of key jumping.  Explained a little later.

Step 2 - Some AD coding for the Super 14 table.

I have already computed my table placings based on the scores that have been entered into the system and the Array is keyed in position order.  In the AD below you will see that I set a counter.  This was initialised to 0 in the initialise program section.  The counter is there so I get the correct record for the correct position in the table as my data driven data only has two records.  If I wanted to, I could have populated more and had the counting already available in the DB1 context as I read each record. Personnal preferences here I guess.  I'd be interested in your viewpoints.

Next, I simply retrieve the data from the Array and do some populating of the fields in the subfile.  Now because I am reading down the data driver file and there are only two records I simply have to reset the cursor by doing a re-read of the first record if I think that there are other records to be displayed.  If not, the second record is read, not a lot happens and the subfile loading has done its job.



This is a screenshot of the device design for the Super14 table.



And the final table with 2009 results in place is as follows:-




That's it.   Simple.  If there are other areas in 2e you'd like me to cover.  Drop me a line.
Thanks for reading.
Lee.

Friday, November 20, 2009

Implementing a 'Generic' Data Driver File + Printing/Displaying Arrays in Subfiles (Part II)

The greatest rugby competition on the planet. Alright, I live in the Southern Hemisphere now and as a direct result have begun to believe the hype. That said the Super 14 (Super 15 from 2011) competition is recognised as one of the strongest leagues in Rugby Union and has team from Australia, South Africa and my current place of residence, New Zealand. Sorry to all those that think I have sold out and not created a Football (Soccer) or NFL example.

System Overview

The requirement was to build a system that allowed the user to make simple sports results/margins predictions on a group of games on a weekly basis. The fuxtures would be published and the predictions made. Once the results were known they would be entered in to the system and participants points (awarded for correct or near correct predictions) would be calculated.

Requirement

Not everyone had the time to trawl the internet looking for a league table that may assist them with making their predictions (Hopefully they have enough time to read this article though). The requirement was to show a realtime league table as fixtures results were entered.  It was decided to record the points the actual teams achieved for each fixture and then simply build the league table on the fly.

Additional information

One could have built a simple file and recreated each and every time the results changed. However, due to the limited number of teams and fixtures in a period it was decided to build on the fly in an array. Also this also meant there was no physical file to maintain and promote and the user could easily view any of the previous years.

Solution

Using a generic data driver file. Build an array that computes and sorts the team table into the correct order and then read from the array the teams and show in that order they are in the table.

Next week I will show you how this was achieved. There will be otherways to receive the same result and all notes are intended as a guide. Your individual circumstances and requirements may vary but feel free to emulate and utilise.

As an appetiser the screen below is a DSPFIL based over an Array.



If you require any further assistance you can always email me at (leedare at talk21 dot com)

Thanks for reading.
Lee.

Tuesday, November 17, 2009

Implementing a 'Generic' Data Driver File + Printing/Displaying Arrays in Subfiles (Part I)

This is a three part story.

I can think of quite a few occassions in 2E where I have needed to display or print information from a non 2e standard file i.e. A non 2E defined file, an array or even a Data Queue.

I have also had the need to build PRTFIL's and DSPFIL's which needed to aggregate data in a master/detail arrangement. The example below is from a change management application I worked on years ago.  It shows a diary note (Header) and the detailed comments (Detail) in one screen and uses a toggle button to determine the entires shown for either the summary mode or detail mode.




To implement these solutions I have used the 'Generic Data Driver' file concept.  I have introduced this at the last 3 2E sites I have worked at.

A worked example of how to do this with screen shots and code sample code will be in part III .  I have also included some notes to help you set up your own generic data driver file and one example of how to utilise it.  This example also has the added bonus of showing you how you can show arrays on a DSPFIL.  Whooarah....Yippee....Get on with it..... I can hear you all say........

This might save Rory and Simon some hassle anyway!!!!  At least with fending off this often requested enhancement to the base 2E tool.

Until then.....(Next Week).

Thanks for reading.
Lee.

Tuesday, February 24, 2009

*Arrays can be quirky in 2e

Hiya,

I have just become aware that *Arrays do not support the correct ordering sequence for negative index values. This has been referred to CA Support (2nd Level) for investigation.

My scenario is an array that is ordered based on a difference between two values. For the purposes of a meaningful example lets pretend that our array is a league table for the English football premier league (Soccer to some). If your game is rugby or another sport then you can draw your own comparisons.

The scenario is that after 2 games of the season I have 5 teams on 4 points. These teams are place 1 to 5 on the table. Let's further embelish this example and assume that my team, Tottenham Hotspur (Spurs) are at the top. :-)

TeamPointsGD (Goal Difference)
Tottenham4pts+78
Liverpool4pts+5
Everton4pts0
Wigan4pts-4
Chelsea4pts-8
......
Arsenal0pts-78


Apart from the obvious good start by Wigan and the strange GD for two games. I believe the example table to be a fair reflection of the real world. With Tottenham at the top. COYS. Blue and White army. Stand up if you hate Arsenal.

If I were to create an *array in DESCENDING order with the keys of Points and GD. My array would sort itself as follows:-

TeamPointsGD (Goal Difference)
Tottenham4pts+78
Liverpool4pts+5
Everton4pts0
Chelsea4pts-8
Wigan4pts-4
......
Arsenal0pts-78


The arrays doesn't handle the negative sign and although it preserves the negative sign it is unable to sort it. Note the order of Chelsea and Wigan.

Until this is fixed, a simple workaround I have used is to *ADD an arbitary figure to the GD to ensure it is a positive value. In order not to blow a limit (as over a season a team can be -100) I need to cater for a higher number so I chose 10,000 for the *array as an offset.

At the point of display which happens to be a DSPFIL I simply deduct 10,000. Simple workaround. Hopefully, simple solution that will be fixed some time in the future. Another option which I contribute to my colleague Chris Koloszar is to do the 10,000 offset for the key and leave the original value as an attribute of the array also.

My main concern is for those of you that have negative values but have yet to discover them.

I will post updates as I hear back from CA.

Thanks for reading.
Lee.

UPDATE HISTORY
==============

2:54pm (Same Day). I have had some quick responses from CA (Very impressed - Thanks Lynn). CA claim this to be working as designed. I am countering that it is a bug and was designed incorrectly. I hope that this will be fixed and I will keep you all updated.

What do you think?

Next Day - Referred to development not a trivial fix but I am confident it will be a good look over. Thanks.