Should COBOL be lockedUP for good?...

Hello EveryOne,

I am developing a softWare application for a multiUser operation
in a netWork environment, the software should facilitate dataEntry
functions, and provide capabilities for an information system
as well. Also, the program has to handle concurrent Read-Write
intensive requests, withOut any bottlenecks or performance blips.

However, I realize that shared dataFiles are indispensable in such
an environment and with such an application, and consideration
must be put forward to correctly create and design sharedFiles,
vis-a-vis fileHandling mechanism.

So, I got the source code cut out with some moot questions that
I would like to share with you, hopefully you can shed some lights
on the layOut, limits and pitfalls, when developing multiTasking
and multiUser softWare programs.

1). Should shared dataFiles use records locking schemes?

2). Should shared dataFiles use data compression mechanism?

3). Should shared dataFiles (largeSize) be split into several
smaller files?

4). Should shared dataFiles be handled by an I/O dynamically
called module per file, that service all processes running
concurrently?

5). Should shared dataFiles be handled by an I/O independent
module per file, that runs separately and communicates with
the application thru shared memory, and service all processes
running concurrently?


Regards, Kellie.

> I think I'd drive the locking more actively than waiting for
> system-supplied timeouts.

That seems to be primarily a Windows deficiency, or at least a
deficiency of Fat Client using shared files.  MS-DOS based systems were
even worse as there seemed to be no timeout at all.

Unix had no record locking mechanism until Xenix was added so compiler
systems built their own locking (or used C-ISAM's).

I mainly used DRI multiuser systems for many years, starting with MP/M
II and MF CIS Cobol + FileShare the Level II and Cobol/2 on
Concurrent-DOS. These gave OS level record locks that were reliable and
released immediately if the process stopped or crashed.

Before I moved to Linux I tested to ensure that automatic unlocking
occurred reliably and quickly.  Running a small number of systems on
Windows (granted it was long ago) had demonstrated that if a client
machine is switched off or rebooted, or the 'close' tag clicked, while
it held a lock invaribly resulted in eventually  having to reboot the
whole network (well at least the file server).

I don't think that 'Timeouts' are sensible. There may be good reasons
why I would want to lock a record for some hours, there are certainly
good reasons why I want all locks released immediately if the client
dies.

> Anyway, why go to all that trouble when an RDBMS will handle it all
properly?

You raise a good point.  What exactly does happen on Windows if I do a
'SELECT .. FOR UPDATE' and obtain a lock and then turn simply pull the
network cable from my client machine ?

Would it remain locked awaiting my next instruction ?  Would it be
polling the client machine to see if I was still there ?  or would it
.... timeout ?

Kellie, you had asserted that:
 

Now we find that "a well designed cobol module with a good algorithm
functions, can do the job very well as needed".

In other words you can write your own which may, or may not, be
'excellent' depending on how good the programmer is. But it is not
_COBOL_ that does this, and it may not be 'any type of file'. How well
does COBOL deal with .xls ? or .doc ? so what does 'any type of file'
actually mean ?

> My compiler (NetExpress) provide a file-sharing and an exellent
locking mechanism as a
> built-In features.

You may start to call it 'excellent' after you have demonstrated how it
behaves when, say, the client machine is powered down after having
locked some records in the server.

You may also need to demonstrate that COMMITT and ROLLBACK work as may
be expected (ie are not treated as comments) before calling it any more
than merely adequate.

> However, regarding the "Restart" mode, perhaps you can give me some
more
> information, or highlight the question?

A 'Restart' is where a program has been updating a file and the program
crashes, or otherwise fails to complete, due to, say, power fail or
data error.  The 'restart' would continue processing from the failure
point or from a checkpoint a short time beforehand, thus saving having
to do a restore and rerun everything up to that point.

With DBMS systems it is relatively easy to emulate restarts using
transactions and commit. File based systems make this more complex.

You may also want to contemplate what happens when the server goes down
(bluescreens or powers down). Certain disk blocks may. or may not, have
been actually written to the disk. They may have been in the client's
network cache, or the server's disk cache or the drive's buffer.  As
there is no guarantee that disk blocks are written in any specific
order an index may have been written but the record not, or vice versa.

Hi Pete,

My sainted mother said:

"The secret of walking on water is not watching your feet.".

Personally, I don't have any negative feeling regarding the use of
dataBase's file
system. I never said in this thread that we should destroy, vanquish,
annihilate
or even ostracize every dataBase management system.

Nonetheless, you are raising a superb point regarding COBOL's file
system
(indexed, relative, sequential), as opposed to the dataBase's file
system (tables),
which I think is a matter of debate, simply because, just because
someOne thinks
that COBOL's file system is not suitable or practical in a multiUser
softWare
application, is no reason to become doctrinaire.

Access to corporate data or informations is an essential part of
information system
technologies, and I am very aware of that fact. As a matter of fact, I
have developed
a COBOL module that I called SnapShot, which functions as an automatic
dataLink to all the files in the multiUser application -- if a user
request a copy of
a file (table to you), the module will ensure that all files that
correlates to the
requested file, will also be generated and copied in plain Text
formats, flat files are
very universal and can be used to alleviate concerns for data access.

Records Locking schemes can be considered a mineField by many skillfull
professional programmers, and some of them will not attempt to use
locks, unless
they are dealing with multiple file servers in a complex netWork
configuration.
however, if I decide to use locking schemes in my softWare programs, my
personal
choice will be to use a record version number, along with a
Date-And-Time stamp,
shared memory and the win32 APIs CreateFileMapping(),
OpenFileMapping(),
MapViewOfFile().

The rollBack command in a dataBase management system simply will undo,
any
changes made to the database by the current transaction on the current
connection, nothing too fancy about this function at all --- I will
create a recover
cobol module that can assists the end-user, to Undo the changes or any
faux pas
they made to a record, in the current session or even later on after
they have
closed the file. I really think the rollBack function should be dynamic
as well as
permanent if it needs to be.

COBOL was declared dead about 40 years ago, put in a coffin, publicly
viewed in
a church and then cremated, when the programming language PL/1 (NPL)
was
introduced in the U.S.A. One can ask, where is PL/1 today?

I am maybe young, crazy, inexperienced and certainly non-conformist ---
however,
my philosophy in life is very simple: Never Follow.

Regards, Kellie.

Kellie Fitton wrote:
>
> 1). Should shared dataFiles use records locking schemes?

Sure.

>
> 2). Should shared dataFiles use data compression mechanism?

Ummm.... Maybe. In addition to the obvious advantages/divantages,
compressed data is impossible to debug and cannot be "read" by an external,
generic program.

>
> 3). Should shared dataFiles (largeSize) be split into several
>    smaller files?

Again, maybe. What's "large?" In the best case your trading (how much)
programming and error potential for (how much) execution efficiency? Three
months of additional programming and debugging effort vs saving thirty
seconds per day probably is not worth it.

>
> 4). Should shared dataFiles be handled by an I/O dynamically
>    called module per file, that service all processes running
>    concurrently?

Probably not.

>
> 5). Should shared dataFiles be handled by an I/O independent
>    module per file, that runs separately and communicates with
>    the application thru shared memory, and service all processes
>    running concurrently?

I don't know.The issue has never come up.

>
>
> Regards, Kellie.

> My sainted mother said:

> "The secret of walking on water is not watching your feet.".

It avoids noticing that it is not working and thus having to admit
failure, if only to yourself.

> I never said in this thread that we should destroy, vanquish,
annihilate
> or even ostracize every dataBase management system.

Strawman. No one said that you had.  You had been quite dismissive
though.

> just because someOne thinks that COBOL's file system is not suitable
or practical
> in a multiUser softWare application, is no reason to become
doctrinaire.

Strawman, no one had said that.

> I have developed a COBOL module that I called SnapShot, which
functions as an
> automatic dataLink to all the files in the multiUser application --

Presumably, based on your earlier claims, hand coded to recognise all
the links for a particular set of files.

> flat files are very universal and can be used to alleviate concerns
for data access.

Flat files are often meaningless. They need descriptions of what the
data items mean and some mechanism for sifting out the required
information.  Usually this means loading them into something that can
manipulate the data, such as a database.

> Records Locking schemes can be considered a mineField by many
skillfull
> professional programmers, and some of them will not attempt to use
locks, ...

Complete nonsense, provide references.

> if I decide to use locking schemes in my softWare programs, my
personal
> choice will be to use a record version number, along with a
Date-And-Time stamp,
> shared memory and the win32 APIs CreateFileMapping(),
> OpenFileMapping(), MapViewOfFile().

Great. Now why did you bother asking questions here ?

> I will create a recover cobol module that can assists the end-user,
to Undo the
> changes or any faux pas they made to a record, in the current session
or even later
> on after they have closed the file

You've never actually written a multiuser system have you, I can tell.

Still, as long as you already have all the answers I am sure that will
have the success that you deserve.

Hi Michael Russell,

> An excellent locking mechanism? How about an excellent
> unlocking mechanism, as well?

If I understand your question correctly, locking and unclocking with
Micro Focus's
COBOL compiler (Net Express), is pretty much descriptive code -- read
with lock,
unClock file-name {record}.

> Anyway, why go to all that trouble when an RDBMS will handle
> it all properly?

Well, just like you said in your reply: the benefit of Isam + COBOL is
tried & trusted.
great, this is music to my ears.

> but, if that's so, how come the fuzziness about timeouts?

according to the manual of Micro Focus, locking and unLocking in a
netWork
environment, is operating system dependent, and requires the support of
the low-level
interrupt h"21" and function h"5C" jointly, even the maximum number of
locks that
the application can acquire, is determined by the OS per se. I am
guessing you might
need to downLoad some service packs, fix patches or some upDates for
your OS.

Regards, Kellie. ;---)

> 1). Should shared dataFiles use records locking schemes?

Absolutely.  Though different techniques can be used for different
applications and different business needs.

For example in an oin-line order entry system it needs to be decided
how the 'race' for stock should be handled.  If there is only a small
number of products then you cannot afford to lock the stock records
over a data-entry interaction because there will be too many clashes.
If there may be stock shortages over many different products, however,
then the stock record can be locked while the available stock is held
awaiting quantity to be entered.

You should also consider using a locking hierarchy. For example it may
be that to work on debtor's transactions every program that does so
must first lock the debtors' master record.  If all correctly follow
the rule then a requirement for multiple locks (kept locks) can be
avoided.

Also ensure that you do specific UNLOCKs when finished.  Typically you
may read through a section of records, such as all order lines for an
order.  It is necessary to read until the order number changes, there
is a risk that the program will leave a lock on the first record of the
next order.

> 2). Should shared dataFiles use data compression mechanism?

Compression probably will use less disk space (it need not) and this
means fewer disk transfers to access a set of data which may make
processing faster.

> 3). Should shared dataFiles (largeSize) be split into several
>    smaller files?

Depends on the value of 'large'.  If I thought that in a few years the
data file may get to be a Gigabyte then I may consider using some
sectioning of data.

> 4). Should shared dataFiles be handled by an I/O dynamically
>    called module per file, that service all processes running
>     concurrently?

That depends on the architecture of the system.  Are you, for example,
going to get just one 'dynamically called module' or will you actually
get an iteration of the module for each run-unit (with or without
shared code) ?

Generally you won't get a _CALLed_ routine to 'service all processes'.


Perhaps you are asking if you should have a separate 'file server'
run-unit that interacts with the programs using IPC in some way, as in
Q5.

That will depend on how you need to balance your network traffic. For
example in a networked system with the programs running on client
machines if a search is to be done by examining each record then the
_whole_ file must be sent across the network one record at a time
(plus, probably, thousands of index blocks).

If a server program runs on the file server and this does searches then
only the 'found' records are sent to the client program.

However, if the architecture of the system is centralised where all the
programs run on an 'application server' accessed via terminal systems
(eg X or putty to a Linux application or RCA clients to a TSE server)
then all disk traffic is local within the box and only screen images go
accross the network.

> 5). Should shared dataFiles be handled by an I/O independent
>     module per file, that runs separately and communicates with
>     the application thru shared memory, and service all processes
>     running concurrently?

'Shared memory' implies that all the run-units are in one machine. In a
client-server situation you need to use IPC that is _not_ shared
memory.

> It does ? How so ? What _COBOL_ features actually do this ?

> it is very easy to BackUp or Restore any indexed or text files on a
> file server machine,
> simply by using a win32 API function, case in point:

So it isn't, as you claimed, a _COBOL_ feature, you are using a Windows
API.

First this is non-portable, it only works on Windows.  It also seems to
be needed to be done on the server.  For some systems you would need to
make two calls, one for the data file and one for the index.

> Also, to Archive any indexed file on the file server, the cobol
> programmer would code
> a module to process some transactions files, and generate an archive
> file as the
> byProduct of that process, and copy the newly created archive files
> into the assigned
> location on the server. Very simple and highly effective process.

Well, yes, you can write a program to do most things. The point being
that you _have_ to write a program - or write additional code into a
program. That can be done in _any_ language. You claimed that "_COBOL_
does an excellent job" when it turns out that it is a programmer that
must do an 'excellent job' and this could be in _any_ language - or
using any API.

But, of course, you haven't even started to address the actual real
problems associated with backup, restore and archiving.

For example: ensuring that the file(s) are not being used when you
attempt to copy them, ensuring that a consistent and complete set is
saved away, ensuring that it is not corrupt before you bother copying
it (and thus overwriting the previous backup).

Kellie,
I won't speak for Richard, but the reason that I questioned your original
statement was that these features are "built-in" to many (most?) RDB product
s.
This is the difference - they are NOT built-in to COBOL file systems (althou
gh
SOME vendors provide such facilities).  As Richard indicates, a programmer
certainly CAN create such code - and in many (some? most?) cases, they will 
do
it "well".  However, the tools provided with most "production quality" datab
ase
systems (RDB - or otherwise) are heavily tested and "production quality"
*before* the COBOL application programmer needs to rely on them.

--
Bill Klein
wmklein <at> ix.netcom.com
"Richard" <riplin@Azonic.co.nz> wrote in message
news:1116316421.600466.285680@g47g2000cwa.googlegroups.com... 
> 
>
> So it isn't, as you claimed, a _COBOL_ feature, you are using a Windows
> API.
>
> First this is non-portable, it only works on Windows.  It also seems to
> be needed to be done on the server.  For some systems you would need to
> make two calls, one for the data file and one for the index.
> 
>
> Well, yes, you can write a program to do most things. The point being
> that you _have_ to write a program - or write additional code into a
> program. That can be done in _any_ language. You claimed that "_COBOL_
> does an excellent job" when it turns out that it is a programmer that
> must do an 'excellent job' and this could be in _any_ language - or
> using any API.
>
> But, of course, you haven't even started to address the actual real
> problems associated with backup, restore and archiving.
>
> For example: ensuring that the file(s) are not being used when you
> attempt to copy them, ensuring that a consistent and complete set is
> saved away, ensuring that it is not corrupt before you bother copying
> it (and thus overwriting the previous backup).
>