field validation (was Re: COBOL/DB2 Date edit question)

>>> On 8/12/2007 at 8:59 PM, in message
< oMednRMwtNq6VCLbnZ2dnUVZ_qmlnZ2d@comcast
.com>, LX-i<lxi0007@netscape.net>
wrote:
>
> There are benefits to stored procedures - but the RDBMS runtime would
> still have been charged to our program, as the thread that started the
> whole mess would have been part of the program's charged time.  :)
> There's another school of thought that stored procedures tie one to a
> particular database, and that a program-based solution is preferable to
> obtain DB independence.  Of course, we've had enough debate in here as
> of late...  ;)
>
> Personally, I believe that call has to be made for each system, by
> analyzing what it's going to do.  On our Unisys mainframe-based COBOL
> system, putting stored procedures in their proprietary RDMS format would
>
> probably make sense.  If it gets ported to another architecture, they're
>
> likely going to have different ways of validating dates.  On the other
> hand, on my websites I'm working on now with PHP 5, I'm using PHP Data
> Objects (PDO), which should allow me to extract the data access from the
>
> underlying database.  (Not that I plan on running it on anything but
> MySQL; but, I'll have the option.)

This brings to mind an interesting issue that we've been struggling with for
years at our shop.  This issue being, where does input validation belong?

Prior to the mid-1990s all of our data input to the mainframe was your good
ol' 3270 green screens.  For better or for worse, presentation logic and
business logic were implemented in the same CICS program.  Data validation
was hard coded directly in to the same CICS program.

We then started utilizing some distributed (Windows) applications to perform
some of these functions.  The same (if slightly modified) CICS programs
still existed on the back end and were utilized by the distributed
applications to actually get the data from there to here (the mainframe).
All of the existing data validation remained in the CICS programs, but much
of the same logic was also put in to the distributed application.

One of the main reasons this was done was because the distributed
application has all those wonderful modern things such as drop-down lists,
radio buttons, etc.  In order to present the proper widgets (or whatever
they're called) to the end user the distributed application must, of course,
know what those valid values are.  Additionally, with those types of
applications the users are used to knowing pretty much immediately of they
have selected a valid option.  They don't want to go through all of the
effort of entering data, then at the end pressing enter and finally getting
back a message "such and such a value is invalid for this account type".

Just this year we implemented a project to 'web enable' a lot of our CSR
screens.  Again we ended up duplicating a lot of the
edits/validations/whatever you want to call them.  In some situations where
the logic is fairly complex and the relevant data resides on the mainframe
we will let things 'pass through' to be edited on the mainframe, but that's
fairly rare.

Another issue we come upon, even if we wanted to pass the edits through to
the mainframe, is that different applications sometimes allow for different
edits.  Here's a real example...  If an ATM card is ordered for a customer
who is a minor (under 18) there is a requirement that the order be approved
by an officer.  So the order is put in to a 'pending approval' queue.  If,
however, the card is ordered as part of setting up a new account through the
new accounts system (a distributed application) then the approval is *not*
required.  So I had to add logic to my CICS program to be able to
distinguish between the transaction being initiated directly (via a regular
3270 terminal session) and same transaction being initiated via a
distributed application (which actually still uses a regular 3270 terminal
session, but a 'psuedo signon' process is done by the distributed
application that lets the CICS program know that it's, well, a distributed
application).

Anyway, there are other cases where the 'end' application is more likely to
know what edits need to be in place, and to 'end' applications may not have
the same edits.  So putting the edits on the mainframe just makes things
worse.

Then again, most of the 'end' applications would have the *same* edits, and
so we end up with a lot of duplicate logic.  Even if the logic is somehow
combined, in, say, a Java object, that object cannot be utilized by a
regular Windows application, and certainly not by a mainframe application
(or mainframe OS does not support Java).

In the end we do what we have to do to make things work.  But no one is
really happy with it.

I don't know that we're going to change our direction at this point, but I
am curious about other's experiences.

Frank

On Mon, 13 Aug 2007 11:26:01 -0600, "Frank Swarbrick"
<Frank.Swarbrick@efirstbank.com> wrote:
 
>< oMednRMwtNq6VCLbnZ2dnUVZ_qmlnZ2d@comcast
.com>, LX-i<lxi0007@netscape.net>
>wrote: 
>
>This brings to mind an interesting issue that we've been struggling with fo
r
>years at our shop.  This issue being, where does input validation belong?
>
>Prior to the mid-1990s all of our data input to the mainframe was your good
>ol' 3270 green screens.  For better or for worse, presentation logic and
>business logic were implemented in the same CICS program.  Data validation
>was hard coded directly in to the same CICS program.
>
>We then started utilizing some distributed (Windows) applications to perfor
m
>some of these functions.  The same (if slightly modified) CICS programs
>still existed on the back end and were utilized by the distributed
>applications to actually get the data from there to here (the mainframe).
>All of the existing data validation remained in the CICS programs, but much
>of the same logic was also put in to the distributed application.
>
>One of the main reasons this was done was because the distributed
>application has all those wonderful modern things such as drop-down lists,
>radio buttons, etc.  In order to present the proper widgets (or whatever
>they're called) to the end user the distributed application must, of course
,
>know what those valid values are.  Additionally, with those types of
>applications the users are used to knowing pretty much immediately of they
>have selected a valid option.  They don't want to go through all of the
>effort of entering data, then at the end pressing enter and finally getting
>back a message "such and such a value is invalid for this account type".
>
>Just this year we implemented a project to 'web enable' a lot of our CSR
>screens.  Again we ended up duplicating a lot of the
>edits/validations/whatever you want to call them.  In some situations where
>the logic is fairly complex and the relevant data resides on the mainframe
>we will let things 'pass through' to be edited on the mainframe, but that's
>fairly rare.
>
>Another issue we come upon, even if we wanted to pass the edits through to
>the mainframe, is that different applications sometimes allow for different
>edits.  Here's a real example...  If an ATM card is ordered for a customer
>who is a minor (under 18) there is a requirement that the order be approved
>by an officer.  So the order is put in to a 'pending approval' queue.  If,
>however, the card is ordered as part of setting up a new account through th
e
>new accounts system (a distributed application) then the approval is *not*
>required.  So I had to add logic to my CICS program to be able to
>distinguish between the transaction being initiated directly (via a regular
>3270 terminal session) and same transaction being initiated via a
>distributed application (which actually still uses a regular 3270 terminal
>session, but a 'psuedo signon' process is done by the distributed
>application that lets the CICS program know that it's, well, a distributed
>application).
>
>Anyway, there are other cases where the 'end' application is more likely to
>know what edits need to be in place, and to 'end' applications may not have
>the same edits.  So putting the edits on the mainframe just makes things
>worse.
>
>Then again, most of the 'end' applications would have the *same* edits, and
>so we end up with a lot of duplicate logic.  Even if the logic is somehow
>combined, in, say, a Java object, that object cannot be utilized by a
>regular Windows application, and certainly not by a mainframe application
>(or mainframe OS does not support Java).

Java exists for both z/OS (unfortunately for you, probably not for
z/VSE) and Windows.
>
>In the end we do what we have to do to make things work.  But no one is
>really happy with it.
>
>I don't know that we're going to change our direction at this point, but I
>am curious about other's experiences.
>
>Frank

>>> On 8/13/2007 at 2:03 PM, in message
<q5e1c3dfemm1rrgrtahacviachdce8dmn4@4ax.com>, Clark F
Morris<cfmpublic@ns.sympatico.ca> wrote:
> On Mon, 13 Aug 2007 11:26:01 -0600, "Frank Swarbrick"
> <Frank.Swarbrick@efirstbank.com> wrote: 
> and 
>
> Java exists for both z/OS (unfortunately for you, probably not for
> z/VSE) and Windows.

Sorry, that was a typo.  I mean "our mainframe OS" (which is VSE and does
not support Java), not "or mainframe OS".

I know Windows (of course!) supports Java.  I have no idea if the Windows
guys would want to call Java routines.  As a one off thing, maybe.  As a
general rule?  I'm guessing not.

Frank

In article <46C03FC9.6F0F.0085.0@efirstbank.com>,
Frank Swarbrick <Frank.Swarbrick@efirstbank.com> wrote:

[snip]

>This brings to mind an interesting issue that we've been struggling with fo
r
>years at our shop.  This issue being, where does input validation belong?

The users may say 'Data validation belongs where it causes response-time
to be fastest.'

The programmers may say 'Data validation belongs where it causes changing
code to be the easiest.'

Both of these may be considered as vectors; data validation, then belongs
at the point where these two vectors intersect with a third one, said
third vector being voiced by the one who says 'I sign the checks for the
system; data validation belongs where it makes me smile the most.'

DD

Frank Swarbrick wrote:
>
> This brings to mind an interesting issue that we've been struggling with f
or
> years at our shop.  This issue being, where does input validation belong?

Our system had it in several places.  The date/time procedure I've
mentioned before (P315) had validation on every routine in it.  We'd
have programs that validated screen input, then passed it to a batch
program that re-validates the same input.  While that may seem wasteful,
it is of note that the batch program could be run without going through
the screen.

When we put a GUI front-end, there were even more edits that happen on
it, before the page is even submitted!  But, again, there are ways to
script things that don't go through the GUI, so a program can't assume
that the input has been validated before.

--
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~~~~~
~     / \/ _ o     ~          Live from Albuquerque, NM!          ~
~     _ /\   |     ~                                              ~
~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
~ Business E-mail  ~ daniel @ "Business Website" below            ~
~ Business Website ~ http://www.djs-consulting.com                ~
~ Tech Blog        ~ http://www.djs-consulting.com/linux/blog     ~
~ Personal E-mail  ~ "Personal Blog" as e-mail address            ~
~ Personal Blog    ~ http://daniel.summershome.org                ~
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~~~~~

GEEKCODE 3.12 GCS/IT d s-:+ a C++ L++ E--- W++ N++ o? K- w$ !O M--
V PS+ PE++ Y? !PGP t+ 5? X+ R* tv b+ DI++ D+ G- e h---- r+++ z++++

"Who is more irrational?  A man who believes in a God he doesn't see,
or a man who's offended by a God he doesn't believe in?" - Brad Stine

X-Trace: individual.net euUOM5/quj2bA9PI+stD8Qhp8/SN186M+farNT3rkAIv0fNF5m
Cancel-Lock: sha1:4lVis8iAOcObCUFv7hShPojCMqo=
X-Priority: 3
X-MSMail-Priority: Normal
X-Newsreader: Microsoft Outlook Express 6.00.2900.3138
X-RFC2646: Format=Flowed; Original
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3138
Bytes: 7082
Xref: number1.nntp.dca.giganews.com comp.lang.cobol:166652



"Frank Swarbrick" <Frank.Swarbrick@efirstbank.com> wrote in message
news:46C08338.6F0F.0085.0@efirstbank.com... 
> <q5e1c3dfemm1rrgrtahacviachdce8dmn4@4ax.com>, Clark F
> Morris<cfmpublic@ns.sympatico.ca> wrote: 
>
> Sorry, that was a typo.  I mean "our mainframe OS" (which is VSE and does
> not support Java), not "or mainframe OS".
>
> I know Windows (of course!) supports Java.  I have no idea if the Windows
> guys would want to call Java routines.  As a one off thing, maybe.  As a
> general rule?  I'm guessing not.
>

I wouldn't hesitate to call Java if the function I needed was available in a
Java bean. (It can be easily brokered through CORBA.)

The way the question is formulated shows the difference in concepts.

You are still thinking in languages and source code; I think in functions
needed, and simply don't care what it's written in. (that's why I have less
resistance to stored DB procedures and triggers; it is just another
language. To me the encapsulation of data and the validation of it, makes it
worth the effort.)

This comes down to component based solutions as a general approach. I have
applications with components written in VB, OO COBOL,  PowerCOBOL, and even
one or two subroutines written in Standard (Fujitsu NetCOBOL) COBOL, and C#,
some of which I wrote and some of which I didn't, (some of which I don't
even have source code for, and don't need it) all playing nicely together
(sometimes even in the same user window) to give a single, seamless, user
experience.

Under Windows this was good, but under DotNET it is superb.

Interop services lets me call unmanaged code into my C# solution, and so I
get to use all of the legacy components I need to, while re-wrapping them
for a modern desktop (or Web) look and feel.

All new development is in C# as it lets stuff run on Windows or Linux  (or
anywhere else that supports DotNET or Mono) without requiring a different
compiler. (A bit like Java really... :-), but, having used both Java and C#,
I have come to prefer C#)

Fortunately, I adopted an n-tier layered approach many years ago so there is
no problem with different looking windows or themes. (Legacy presentation
layer components are simply re-written in C#) This is about 20% of my
legacy, so the remaining 80% that has no visual interface, can be easily
re-used. I COULD re-use some of the Windows (PowerCOBOL, especially) stuff,
but it just looks old-fashioned and dated.)

I'm currently revisiting an application I wrote 7 years ago for converting
ISAM and VSAM/KSDS datasets into RDB. It is being enhanced to generate DDL
for various RDBMS, and the whole thing is being re-wrapped into a single
"Management Console" that lets you control the whole process from a single
point. After refactoring PowerCOBOL and OO COBOL I was amazed to have the
main function (analyzes COBOL source SELECT statements and FD/01 record
definitions, then generates an ACCESS DB (really for documentation, but also
for quick initial testing) and DDL for the RDB of your choice), run
perfectly on the first attempt in the new DotNET environment, using the C#
environment handling, control, and user interface.

When I first wrote this system from scratch it took 5 months; the upgrade
and migration to DotNET will take 3/4 ws with many original components
being re-used without change, just "glued" together differently...New
facilities are also being added. There is a possibility I may be asked to
write some tools to automate the exisiting COBOL code conversion to RDB
also. Any of you who have manually converted code and data to RDB will know
that this is not to be undertaken lightly. It isn't just a matter of
converting ISAM verbs to SQL; these are different paradigms, and OCCURS,
REDEFINES, and group fields in the ISAM definitions have no place on the new
RDB. It all has to be handled.

Once the RDB has been defined and normalized, it needs to be loaded. I can
generate COBOL modules to do that (the client is a COBOL shop) and that was
what led to my question regarding Batch Compiling Fujitsu COBOL (see
different thread here in CLC)

I'm now thinking about NOT generating COBOL but simply generating DDL to
load the data instead. (Of course I still have to read it  somehow, and it
is in the ISAM system which is not readily open to tools other than COBOL.)

The point is that component based systems pay off in ways you can't even
imagine when you build them. I had no idea 7 years ago that I would need to
upgrade and enhance this system in some major ways, 7 years down the track.
Now I'm being paid to do so, and thanking my lucky stars it is not a
complete rewrite...in fact, I can focus on the new facilities rather than
having to worry about complete redesigns and rebuilds of the old ones.

Encapsulation rules; objects are what matters; procedural source is as
relevant as Latin.

(Of course there are enclaves where Latin is still important... Vatican City
springs to mind... :-))

Pete.
--
"I used to write COBOL...now I can do anything."

--
"I used to write COBOL...now I can do anything."
"LX-i" <lxi0007@netscape.net> wrote in message
 news:iO2dnbe7vsZjpVzbnZ2dnUVZ_jydnZ2d@co
mcast.com...
> Frank Swarbrick wrote: 
>
> Our system had it in several places.  The date/time procedure I've
> mentioned before (P315) had validation on every routine in it.  We'd have
> programs that validated screen input, then passed it to a batch program
> that re-validates the same input.  While that may seem wasteful, it is of
> note that the batch program could be run without going through the screen.
>
> When we put a GUI front-end, there were even more edits that happen on it,
> before the page is even submitted!  But, again, there are ways to script
> things that don't go through the GUI, so a program can't assume that the
> input has been validated before.
>

All of which makes "encapsulated back end validation" (stored procedures and
triggers) just that bit more appealing.

Pete.
--
"I used to write COBOL...now I can do anything."

<docdwarf@panix.com> wrote in message news:f9qtvl$sdd$1@reader1.panix.com...
> 
>
> The users may say 'Data validation belongs where it causes response-time
> to be fastest.'
>
> The programmers may say 'Data validation belongs where it causes changing
> code to be the easiest.'

Striking a balance between features, performance and maitainability is a
long-running  battle; rightly so, and rightly should and will continue.

I think there's also some kind of aphorism on this, something like "speed,
features, maintainability: pick two" or something like that.

MCM

In article <h1gwi.873$924.548@newssvr23.news.prodigy.net>,
Michael Mattias <mmattias@talsystems.com> wrote:
><docdwarf@panix.com> wrote in message news:f9qtvl$sdd$1@reader1.panix.com..
. 
>
>Striking a balance between features, performance and maitainability is a
>long-running  battle; rightly so, and rightly should and will continue.

I was trying to be even less specific than that, Mr Mattias, and
attempting to liken the difficulty to that age-old question of 'what
constitutes a Logical Unit of Work?'

Where something 'belongs', I'd say, depends on the criteria one uses to
judge 'belonging'... and, as you point out, different groups have
differing values that they use to weight criteria.

DD

On Wed, 15 Aug 2007 03:22:49 +1200, "Pete Dashwood"
<dashwood@removethis.enternet.co.nz> wrote:
 
>
>No, Howard, editing string length for input fields on web pages is a
>valuable and accepted line of defence  against SQL inection attacks. (It is
>only one measure, but a very important one)
>
>It is a pain (I hate writing it), but in today's world it is a necessary
>evil.

Then allow me to enter my complete Zip code.   Are 9 digit postal
codes more dangerous to enter than are 5 digit postal codes?