Reading CopyBook & Data Files

I'm still trying to extract the data from an old COBOL data base. (Thanks
for the help so far!)

Please check me... and answer a couple of questions...

Two files to consider...
- Owners.CPY is the CopyBook file that defines the fields
- Owners.FIL is the file that holds the actual data

Here is the begining of Owners.CPY...

FD  OWNER-MASTER-FILE.
01  OWNER-MASTER-RECORD.
03  OMR-RECORD-KEY.
05  OMR-OWNER-ID                PIC X(6).
03  OMR-PRIMARY-RECORD-KEY          PIC X(6).
03  OMR-OWNER-NAME.
05  OMR-LAST-NAME               PIC X(20).
05  OMR-FIRST-NAME              PIC X(15).
05  OMR-MI                      PIC X.

My question is this... when I open the corrosponding data file Owners.FIL
there is a file header that I want to discard. I'm looking for the first
byte of actual data here. Is it true that the first three lines in the above
file do not "reserve" any bytes for data... that is to say... the 4th
line...
05  OMR-OWNER-ID                PIC X(6).
Reserves 6 bytes for a record ID... so these should be the first bytes in
the file that actually hold data for the first record...

Therefore I am guessing that everything before this is file hearder and not
associated with any particular record in the table... and thus can be
discarded as I am only interested in getting a file with just the data...

Here is the first fo the data file Owners.FIL (I inserted the "-" for nulls
and other non ASCII characters)
0~------9902151236070899021512360708----------------------------------------
--------------------------------A-------------------A·AAF001------SMITH

at 131 bytes in is the data AAF001
I think this is the 6 bytes defined in OMR-OWNER-ID and see such at the
begining of all subisiquent records
BUT... what about the preceeding 2 bytes containing A·
These two characters also appear at the begining of each record... so this
implies the first 3 lines of Owners.CPY may actually reserve some space in
each record ???

Thanks for any assistance here as I think I'm getting close.

"Will" <Will@somewhere.com> wrote in message
 news:4762b41b$0$2385$4c368faf@roadrunner
.com...
> I'm still trying to extract the data from an old COBOL data base. (Thanks
> for the help so far!)

One word of caution here... the COBOL copy libraries give you a LOGICAL
picture of the data and the PHYSICAL storage in media may not necessarily
match that.

Granted, it *usually* will, but especially with indexed pairs or
variable-length records I'd be real careful to check that there might not be
some "length words" or other control data in the PHYSICAL storage which do
not appear in the File Description.

--
Michael C. Mattias
Tal Systems Inc.
Racine WI
mmattias@talsystems.com

On Fri, 14 Dec 2007 11:49:41 -0500, "Will" <Will@somewhere.com> wrote:

>I'm still trying to extract the data from an old COBOL data base. (Thanks
>for the help so far!)
>
>Please check me... and answer a couple of questions...
>
>Two files to consider...
>- Owners.CPY is the CopyBook file that defines the fields
>- Owners.FIL is the file that holds the actual data
>
>Here is the begining of Owners.CPY...
>
>       FD  OWNER-MASTER-FILE.
>       01  OWNER-MASTER-RECORD.
>           03  OMR-RECORD-KEY.
>               05  OMR-OWNER-ID                PIC X(6).
>           03  OMR-PRIMARY-RECORD-KEY          PIC X(6).
>           03  OMR-OWNER-NAME.
>               05  OMR-LAST-NAME               PIC X(20).
>               05  OMR-FIRST-NAME              PIC X(15).
>               05  OMR-MI                      PIC X.
>
>My question is this... when I open the corrosponding data file Owners.FIL
>there is a file header that I want to discard. I'm looking for the first
>byte of actual data here. Is it true that the first three lines in the abov
e
>file do not "reserve" any bytes for data... that is to say... the 4th
>line...
>05  OMR-OWNER-ID                PIC X(6).
>Reserves 6 bytes for a record ID... so these should be the first bytes in
>the file that actually hold data for the first record...
>
>Therefore I am guessing that everything before this is file hearder and not
>associated with any particular record in the table... and thus can be
>discarded as I am only interested in getting a file with just the data...
>
>Here is the first fo the data file Owners.FIL (I inserted the "-" for nulls
>and other non ASCII characters)
>0~------9902151236070899021512360708---------------------------------------
---------------------------------A-------------------A·AAF001------SMITH
>
>at 131 bytes in is the data AAF001
>I think this is the 6 bytes defined in OMR-OWNER-ID and see such at the
>begining of all subisiquent records
>BUT... what about the preceeding 2 bytes containing A·
>These two characters also appear at the begining of each record... so this
>implies the first 3 lines of Owners.CPY may actually reserve some space in
>each record ???
>
>Thanks for any assistance here as I think I'm getting close.

The first two bytes of each record are the record length. They have nothing 
to do with the
first three lines in the copybook.

The only lines in the copybook that 'take up space' are the ones with PIC.

Where the record description starts with
OMR-RECORD-KEY
AND you have previously indicated there are files like
xxx.IDX

this probably was an INDEXED file.  This means that it is VERY possible that
there is "stuff" between each record as well as data before the beginning of
 the
"real" data.

Also, have you just shown us "simple" versions of the record description or 
are
ALL the fields really defined with 'PIC X".  If any of your real definitions
have "PIC 9" in them, you may well have other problems.

--
Bill Klein
wmklein <at> ix.netcom.com
"Will" <Will@somewhere.com> wrote in message
 news:4762b41b$0$2385$4c368faf@roadrunner
.com...
> I'm still trying to extract the data from an old COBOL data base. (Thanks 
for
> the help so far!)
>
> Please check me... and answer a couple of questions...
>
> Two files to consider...
> - Owners.CPY is the CopyBook file that defines the fields
> - Owners.FIL is the file that holds the actual data
>
> Here is the begining of Owners.CPY...
>
>       FD  OWNER-MASTER-FILE.
>       01  OWNER-MASTER-RECORD.
>           03  OMR-RECORD-KEY.
>               05  OMR-OWNER-ID                PIC X(6).
>           03  OMR-PRIMARY-RECORD-KEY          PIC X(6).
>           03  OMR-OWNER-NAME.
>               05  OMR-LAST-NAME               PIC X(20).
>               05  OMR-FIRST-NAME              PIC X(15).
>               05  OMR-MI                      PIC X.
>
> My question is this... when I open the corrosponding data file Owners.FIL
> there is a file header that I want to discard. I'm looking for the first b
yte
> of actual data here. Is it true that the first three lines in the above fi
le
> do not "reserve" any bytes for data... that is to say... the 4th line...
> 05  OMR-OWNER-ID                PIC X(6).
> Reserves 6 bytes for a record ID... so these should be the first bytes in 
the
> file that actually hold data for the first record...
>
> Therefore I am guessing that everything before this is file hearder and no
t
> associated with any particular record in the table... and thus can be
> discarded as I am only interested in getting a file with just the data...
>
> Here is the first fo the data file Owners.FIL (I inserted the "-" for null
s
> and other non ASCII characters)
> 0~------9902151236070899021512360708--------------------------------------
----------------------------------A-------------------A·AAF001------SMITH
>
> at 131 bytes in is the data AAF001
> I think this is the 6 bytes defined in OMR-OWNER-ID and see such at the
> begining of all subisiquent records
> BUT... what about the preceeding 2 bytes containing A·
> These two characters also appear at the begining of each record... so this
> implies the first 3 lines of Owners.CPY may actually reserve some space in
> each record ???
>
> Thanks for any assistance here as I think I'm getting close.
>
>
>

On Dec 15, 5:49 am, "Will" <W...@somewhere.com> wrote:
> I'm still trying to extract the data from an old COBOL data base. (Thanks
> for the help so far!)
>
> Please check me... and answer a couple of questions...
>
> Two files to consider...
> - Owners.CPY is the CopyBook file that defines the fields
> - Owners.FIL is the file that holds the actual data
>
> Here is the begining of Owners.CPY...
>
>        FD  OWNER-MASTER-FILE.
>        01  OWNER-MASTER-RECORD.
>            03  OMR-RECORD-KEY.
>                05  OMR-OWNER-ID                PIC X(6).
>            03  OMR-PRIMARY-RECORD-KEY          PIC X(6).
>            03  OMR-OWNER-NAME.
>                05  OMR-LAST-NAME               PIC X(20).
>                05  OMR-FIRST-NAME              PIC X(15).
>                05  OMR-MI                      PIC X.
>
> My question is this... when I open the corrosponding data file Owners.FIL
> there is a file header that I want to discard.

I am not sure that you have posted the exact identification of the
Cobol product. It seems most likely to be MicroFocus Cobol/2 version
between 2.1 and 3,4.

These could write several different formats for indexed files. If
there is a 'file header' then this is most likely the C2 format. The
formats are fully explained in the manuals for the product.

With C2 there is a 128byte file header. Each record then has a 2 (or
4) byte record header. You should note that in using an indexed file
records may be rewritten or deleted. The record header indicates the
status of the following record and particular values indicate
'deleted' or a system record or other. Only when the first 4 bits of
the record header contain '0100' is it a user record. If you find a
value here > 0100 (4) then you will find that it is even more
complicated with pointers and reduced data.

Each record header will start on a 4 byte boundary, so up to 3 padding
characters will follow each data record. The other 12 bits of the
record header give the record length as this may be variable, and this
does not include the padding.

> I'm looking for the first
> byte of actual data here. Is it true that the first three lines in the abo=[/color
]
ve
> file do not "reserve" any bytes for data... that is to say... the 4th
> line...
> 05  OMR-OWNER-ID                PIC X(6).
> Reserves 6 bytes for a record ID... so these should be the first bytes in
> the file that actually hold data for the first record...
>
> Therefore I am guessing that everything before this is file hearder and no=[/color
]
t
> associated with any particular record in the table... and thus can be
> discarded as I am only interested in getting a file with just the data...

Maybe, but discarding the headers may mean that you get more than
'just the data' you will also get deleted records and other stuff.

> Here is the first fo the data file Owners.FIL (I inserted the "-" for null=[/color
]
s
> and other non ASCII characters)
> 0~------9902151236070899021512360708--------------------------------------=[/color
]
----------------------------------A-------------------A=B7AAF001------SMITH
>
> at 131 bytes in is the data AAF001
> I think this is the 6 bytes defined in OMR-OWNER-ID and see such at the
> begining of all subisiquent records
> BUT... what about the preceeding 2 bytes containing A=B7
> These two characters also appear at the begining of each record... so this=[/color
]

> implies the first 3 lines of Owners.CPY may actually reserve some space in=[/color
]

> each record ???

No. there two bytes are system generated header. Only the PICs reserve
space in the records.

Under the FD there may be more than one 01 record level (though it
seems unlikely for this file).  Each 01 in a single FD describes the
same record area, they implicitly redefine each other. If there is
more than one 01 then you will need to identify the record to choose
which layout applies.

> Thanks for any assistance here as I think I'm getting close.

If this is MicroFocus then you should have a REBUILD.EXE program.
This can do most of the work for you by extracting the data records
and ignoring the deleted records and headers.

For example if you rebuild to a Level II format then the data file for
that will not have file or record headers

REBUILD OWNERS.FIL,XOWNERS.LII /s:mf /t:lii

The data file (there will also be a separate .IDX) will be a simple
structure consisting of fixed length datarecordCRLF where CRLF is
x"0D0A" for valid records and x"0D00" for deleted or no record.

Will wrote:
> I'm still trying to extract the data from an old COBOL data base.
> (Thanks for the help so far!)
>
> Please check me... and answer a couple of questions...
>
> Two files to consider...
> - Owners.CPY is the CopyBook file that defines the fields
> - Owners.FIL is the file that holds the actual data
>
> Here is the begining of Owners.CPY...
>
>       FD  OWNER-MASTER-FILE.
>       01  OWNER-MASTER-RECORD.
>           03  OMR-RECORD-KEY.
>               05  OMR-OWNER-ID                PIC X(6).
>           03  OMR-PRIMARY-RECORD-KEY          PIC X(6).
>           03  OMR-OWNER-NAME.
>               05  OMR-LAST-NAME               PIC X(20).
>               05  OMR-FIRST-NAME              PIC X(15).
>               05  OMR-MI                      PIC X.
>
> My question is this... when I open the corrosponding data file
> Owners.FIL there is a file header that I want to discard. I'm looking
> for the first byte of actual data here. Is it true that the first
> three lines in the above file do not "reserve" any bytes for data...
> that is to say... the 4th line...
> 05  OMR-OWNER-ID                PIC X(6).
> Reserves 6 bytes for a record ID... so these should be the first
> bytes in the file that actually hold data for the first record...
>
> Therefore I am guessing that everything before this is file hearder
> and not associated with any particular record in the table... and
> thus can be discarded as I am only interested in getting a file with
> just the data...
> Here is the first fo the data file Owners.FIL (I inserted the "-" for
> nulls and other non ASCII characters)
> 0~------9902151236070899021512360708--------------------------------------
----------------------------------A-------------------A·AAF001------SMITH
>
> at 131 bytes in is the data AAF001
> I think this is the 6 bytes defined in OMR-OWNER-ID and see such at
> the begining of all subisiquent records
> BUT... what about the preceeding 2 bytes containing A·
> These two characters also appear at the begining of each record... so
> this implies the first 3 lines of Owners.CPY may actually reserve
> some space in each record ???
>
> Thanks for any assistance here as I think I'm getting close.

No, but you got the door to Mordor open...

Be aware than an indexed file contains LOGICAL data, positioning
information, overflow areas, deleted records, and pointers to other parts of
the file containing more data. These various logical components are mixed up
higgedly-piggedly and tempered with PHYSICAL information (such as record
length, block size, packing percentage, and more).

It is a mind-boggling task to deconstruct an ISAM file. An ISAM file is,
truely, unlike anything you've ever seen.

Far, far easier is to use the vendor-supplied utility to convert the file to
a text file. This vendor-supplied utility knows about all the internal
machinations of the file. If you don't have this utility, or can't find it,
someone here will gladly take your files, massage them, and return the
results.

Bill,

Yes "PIC 9" shows up in some of the Copybook files... like...

05  CDR-CONTRACT-NBR            PIC 9(6).
05  CDR-RECORD-NBR              PIC 999.

But this seems to indicate a number will occupy the assoiated bytes...

For instance...

PIC 9(6). would reserve 6 bytes for numbers
PIC 999. would reserve 3 bytes for numbers

Right?



"William M. Klein" <wmklein@nospam.netcom.com> wrote in message
news:ccA8j.157856$_H4.123946@fe08.news.easynews.com...
> Where the record description starts with
>    OMR-RECORD-KEY
> AND you have previously indicated there are files like
>    xxx.IDX
>
> this probably was an INDEXED file.  This means that it is VERY possible
> that there is "stuff" between each record as well as data before the
> beginning of the "real" data.
>
> Also, have you just shown us "simple" versions of the record description
> or are ALL the fields really defined with 'PIC X".  If any of your real
> definitions have "PIC 9" in them, you may well have other problems.
>
> --
> Bill Klein
> wmklein <at> ix.netcom.com
> "Will" <Will@somewhere.com> wrote in message
>  news:4762b41b$0$2385$4c368faf@roadrunner
.com... 
>
>

"HeyBub" <heybub@gmail.com> wrote in message
news:13m6qq972rvbr1e@corp.supernews.com...

> Far, far easier is to use the vendor-supplied utility to convert the file
> to a text file. This vendor-supplied utility knows about all the internal
> machinations of the file. If you don't have this utility, or can't find
> it, someone here will gladly take your files, massage them, and return the
> results.

I think you left off  "for a fee"  didn't you?


MCM

On Sat, 15 Dec 2007 07:53:16 -0500, "Will" <Will@somewhere.com> wrote:

>Bill,
>
>Yes "PIC 9" shows up in some of the Copybook files... like...
>
>               05  CDR-CONTRACT-NBR            PIC 9(6).
>               05  CDR-RECORD-NBR              PIC 999.
>
>But this seems to indicate a number will occupy the assoiated bytes...
>
>For instance...
>
>PIC 9(6). would reserve 6 bytes for numbers
>PIC 999. would reserve 3 bytes for numbers
>
>Right?

Right. If the line says COMP or COMP-(number), that's when you "have other p
roblems".


>"William M. Klein" <wmklein@nospam.netcom.com> wrote in message
>news:c
cA8j.157856$_H4.123946@fe08.news.easynews.com... 
>

Robert, No COMP or COMP-(number) in any of the Copybook files.

I'm posting a seperate update with some questions about parsing the data
files.

Thanks for the help.

"Robert" <no@e.mail> wrote in message
 news:eb08m31279pi95sdib1scdtibd87s7i7v6@
4ax.com...
> On Sat, 15 Dec 2007 07:53:16 -0500, "Will" <Will@somewhere.com> wrote:
> 
>
> Right. If the line says COMP or COMP-(number), that's when you "have other
> problems".
>
> 
> cA8j.157856$_H4.123946@fe08.news.easynews.com... 
>