Home > Archive > Cobol > February 2006 > MicroFocus & mainframe sequential files
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
MicroFocus & mainframe sequential files
|
|
| pottmi@gmail.com 2006-02-08, 8:52 am |
| I have been trying to get a microfocus cobol program to open and read a
variable sequential file that is the same format as what gets transfer
to my computer via ftp.
That transfer format is very simple:
2 bytes indicating the length of the data.
2 bytes of low values
N bytes of data.
that pattern repeated for each record.
Microfocus seems to want a 128 byte header on the file, along with some
other differences.
My questions are:
1) has anyone found a combination of options that allow a cobol program
compiled with the microfocus compiler to read such a file?
2) has anyone written an external file handler using microfocus
sanctioned techniques that will read such a file.
3) has anyone written an external file handler using microfocus
sanctioned techniques that I can have so I can use it as a starting
point to read such a file.
By MicroFocus sanctioned techniques, I am referring to the microfocus
"File Handling" book.
Just to save everyone some time...
I dont need to be told about options like converting the data comma
delimited files for transfer, and I dont need any information about the
possibilities of the data not being compatible. My question is about
sequential file format compatibilty, not data compatiblity. Also, I
have already written a C program to do the conversion so I already have
a backup plan if I am not able to read the files directly.
Michael Potter
| |
| Michael Mattias 2006-02-08, 8:52 am |
| > I have been trying to get a microfocus cobol program to open and read a
> variable sequential file that is the same format as what gets transfer
> to my computer via ftp.
>
> That transfer format is very simple:
> 2 bytes indicating the length of the data.
> 2 bytes of low values
> N bytes of data.
> that pattern repeated for each record.
>
> Microfocus seems to want a 128 byte header on the file, along with some
> other differences.
>
> My questions are:
> 1) has anyone found a combination of options that allow a cobol program
> compiled with the microfocus compiler to read such a file?
If the data truly are a stream like this (that is, there are no 'end of
record' markers in the input stream) then you should be able to read it as a
sequential file with any record length you want, provided once you read a
record you take over the job of figuring out how much of what you read is
actually one logical record, and you can execute any additional reads
required to get the rest of a logical record.
ie., say you pick a record length of 20
Read first record; let's say the length is give you as 10 bytes. Discard
the four length bytes (I'll bet that's actually a 32-bit integer, not a
16-bit integer plus 0x0000), move next ten to your first output record.
Now read the next four bytes of the first record to get the length of the
next data record. Let's say that is also ten bytes. Get the remainder of
your first record (20 - 4 - 10 -4 = 2) and make that the first two bytes of
your second output record. Now read the next 20 bytes. The first eight bytes
will be the rest of logical record number two, and the lengthword+data
follows that.
Nobody said it would be fun.
MCM
| |
| pottmi@gmail.com 2006-02-08, 8:52 am |
| The data is truely as I stated. The second pair of bytes is always low
values, it is not a 32 bit integer. I presume that IBM decided to
leave a pair of extra bytes available incase they ever wanted to
support records larger than 32k. You can not treat them as a 32bit
value even tho they are always zero because the byte order is not
defined. When I read the raw data with C I throw an error if the
second pair have anything but zero.
Back to the problem...
I am trying to read the data using existing COBOL programs without
modifing the existing COBOL programs. I can do this as long as I
convert the data to the microfocus format (w/128 byte header). Now I
am trying to figure out what compile options will allow me to read the
data without converting it to microfocus format first. I would also
consider well defined changes to the code such as adding a "RECORD MODE
VARIABLE" to the select statement.
I already called Microfocus and 1st level support could not help. I am
going to call Monday morning to see if someone in the UK has a
different angle. The compiler has some many options I would not blame
first level support if they did not know of the existence of a
particular compile or runtime option.
thanks for you quick response.
Michael Potter
| |
| James J. Gavan 2006-02-08, 8:52 am |
| pottmi@gmail.com wrote:
> I have been trying to get a microfocus cobol program to open and read a
> variable sequential file that is the same format as what gets transfer
> to my computer via ftp.
>
> That transfer format is very simple:
> 2 bytes indicating the length of the data.
> 2 bytes of low values
> N bytes of data.
> that pattern repeated for each record.
>
> Microfocus seems to want a 128 byte header on the file, along with some
> other differences.
>
> My questions are:
> 1) has anyone found a combination of options that allow a cobol program
> compiled with the microfocus compiler to read such a file?
> 2) has anyone written an external file handler using microfocus
> sanctioned techniques that will read such a file.
> 3) has anyone written an external file handler using microfocus
> sanctioned techniques that I can have so I can use it as a starting
> point to read such a file.
>
> By MicroFocus sanctioned techniques, I am referring to the microfocus
> "File Handling" book.
>
> Just to save everyone some time...
> I dont need to be told about options like converting the data comma
> delimited files for transfer, and I dont need any information about the
> possibilities of the data not being compatible. My question is about
> sequential file format compatibilty, not data compatiblity. Also, I
> have already written a C program to do the conversion so I already have
> a backup plan if I am not able to read the files directly.
>
> Michael Potter
>
I see elsewhere you hope to read in without necessarily going through an
interim conversion - good luck if you can get the appropriate answer
from Tech in UK.
Have you considered the possibility of using M/F Byte-Stream file
techniques using CBL_XXX_FILE routines - that's assuming they apply to
your compiler. Shouldn't be difficult from what you have above, allowing
you to read in chunks, or if the whole file is not too large, read it in
as one big chunk.
If interested I can send you a zipfile of two M/F Demos - e-mail me as
above but "edit" my e-mail address.
Jimmy, Calgary AB
| |
| Colin Campbell 2006-02-08, 8:52 am |
| pottmi@gmail.com wrote:
>I have been trying to get a microfocus cobol program to open and read a
>variable sequential file that is the same format as what gets transfer
>to my computer via ftp.
>
>That transfer format is very simple:
>2 bytes indicating the length of the data.
>2 bytes of low values
>N bytes of data.
>that pattern repeated for each record.
>
>Microfocus seems to want a 128 byte header on the file, along with some
>other differences.
>
>My questions are:
>1) has anyone found a combination of options that allow a cobol program
>compiled with the microfocus compiler to read such a file?
>2) has anyone written an external file handler using microfocus
>sanctioned techniques that will read such a file.
>3) has anyone written an external file handler using microfocus
>sanctioned techniques that I can have so I can use it as a starting
>point to read such a file.
>
>By MicroFocus sanctioned techniques, I am referring to the microfocus
>"File Handling" book.
>
>Just to save everyone some time...
>I dont need to be told about options like converting the data comma
>delimited files for transfer, and I dont need any information about the
>possibilities of the data not being compatible. My question is about
>sequential file format compatibilty, not data compatiblity. Also, I
>have already written a C program to do the conversion so I already have
>a backup plan if I am not able to read the files directly.
>
>Michael Potter
>
>
>
Michael,
I believe MicroFocus took care of your problem many years ago. I looked
at the manuals that came with the MF Workbench V3.1, dated 1993. They
supplied a utility program VRECGEN to be run on the mainframe before
downloading variable length record data, and VRECGEN2, to run on the PC
before going the opposite direction.
Do you have documentation of those programs, and / or can you find them
in the distributed code (in those days, the directory structure was
COBOL\SAMPLE)?
The assumption was that you would use SEND / RECEIVE to move the data
from the mainframe and back to it. VRECGEN "converted" RECFM=V records
so that you could use the resulting records directly in your program on
the PC.
I don't have the old Workbench installed anywhere, nor do I have a
mainframe connection, so I can't test this out. And I haven't seen a
release of MF COBOL for many years, so I don't know whether they've
changed the approach you should take for managing RECFM=V records, but I
do recall that this approach worked satisfactorily.
Final note:
You might want to study the format of variable length records, including
spanned records. It isn't 100% accurate to say that the last half of a
record descriptor word contains low-values.
You can find the exact format information by Googling on "IBM Using Data
Sets". (There is a manual by that name.) For spanned records, search
that manual for "segment control code".
| |
| Michael Mattias 2006-02-08, 8:52 am |
| <pottmi@gmail.com> wrote in message
news:1139101225.085526.164350@g44g2000cwa.googlegroups.com...
>
> Back to the problem...
>
> I am trying to read the data using existing COBOL programs without
> modifing the existing COBOL programs. I can do this as long as I
> convert the data to the microfocus format (w/128 byte header).
This truly IS a problem.
You have an existing program designed to read data provided in a certain
format; you have data in another format; there is no way you are going to
get this to work. This is a dream and you WILL wake up.
You need to either convert the data to the expected format, or change the
format in which the existing program expects the data.
Others have made some good suggestions to do the former; that's that path
down which I'd set out.
MCM
| |
| pottmi@gmail.com 2006-02-08, 8:53 am |
| I know that I can convert the data to make the problem "go away", in
fact I am already doing that as an (hopefully) interim measure. Adding
the conversion step increases IO and I want to avoid the IO by having
the COBOL program read the data without a conversion.
I tried to make that clear in my original post so no one would have to
waste their time answering a question that I did not ask.
By conversion, I mean the format of the meta data in the sequential
file, not the data itself. By meta data I mean the 128 byte header
record that microfocus requires, the header on each record that
indicates length, and sometimes padding bytes to bring new records to
the appropriate byte boundary.
--
potter
| |
| pottmi@gmail.com 2006-02-08, 8:53 am |
| Thanks Colin,
I think I will stick with my own conversion program as I will not have
to run anything on the mainframe to make it work. Plus, I am using
microfocus SE on AIX.
Your "Final note" is quite helpful I will interested in finding out
what those extra bytes are used for. I have alway been curious, but
never had a need to figure it out.
| |
| pottmi@gmail.com 2006-02-08, 8:53 am |
| I am not really considering changing the COBOL programs, there are
about 400 of them reading about 400 different sequential files. My
worst case right now is having to run my conversion program on the data
before the program runs.
Never the less, I am interested in your sample programs. I will send a
separate email to you so all you have to do is reply.
| |
|
| In article <1139180224.894240.81710@g44g2000cwa.googlegroups.com>,
<pottmi@gmail.com> wrote:
>I know that I can convert the data to make the problem "go away", in
>fact I am already doing that as an (hopefully) interim measure. Adding
>the conversion step increases IO and I want to avoid the IO by having
>the COBOL program read the data without a conversion.
1) What is the anticipated transaction volume involved (millions of
records per unit time)?
2) What controls exist to insure that the format does not change, again,
as it is changing now?
>
>I tried to make that clear in my original post so no one would have to
>waste their time answering a question that I did not ask.
The fact that you did not ask it does not necessarily generate the
conclusion that the question you did not ask is irrelevant.
DD
| |
| William M. Klein 2006-02-08, 8:53 am |
| Read the Micro Focus documentation on:
- VRECGEN
and/or
- RDW directive
Both of these relate specially to your "problem" and tell you what is and isn't
supported.
--
Bill Klein
wmklein <at> ix.netcom.com
<pottmi@gmail.com> wrote in message
news:1139086819.267599.327500@g47g2000cwa.googlegroups.com...
>I have been trying to get a microfocus cobol program to open and read a
> variable sequential file that is the same format as what gets transfer
> to my computer via ftp.
>
> That transfer format is very simple:
> 2 bytes indicating the length of the data.
> 2 bytes of low values
> N bytes of data.
> that pattern repeated for each record.
>
> Microfocus seems to want a 128 byte header on the file, along with some
> other differences.
>
> My questions are:
> 1) has anyone found a combination of options that allow a cobol program
> compiled with the microfocus compiler to read such a file?
> 2) has anyone written an external file handler using microfocus
> sanctioned techniques that will read such a file.
> 3) has anyone written an external file handler using microfocus
> sanctioned techniques that I can have so I can use it as a starting
> point to read such a file.
>
> By MicroFocus sanctioned techniques, I am referring to the microfocus
> "File Handling" book.
>
> Just to save everyone some time...
> I dont need to be told about options like converting the data comma
> delimited files for transfer, and I dont need any information about the
> possibilities of the data not being compatible. My question is about
> sequential file format compatibilty, not data compatiblity. Also, I
> have already written a C program to do the conversion so I already have
> a backup plan if I am not able to read the files directly.
>
> Michael Potter
>
| |
| pottmi@gmail.com 2006-02-09, 6:55 pm |
| Here is how I solved the problem of reading mainframe style ESDS files
with RDWs directly using MicroFocus COBOL:
I wrote a new file handler and used the microfocus compile option
called EXTFH to install my file handler in place of the microfocus File
handler. I wrote it in C because I could reused the routines from my
conversion program to do the reading and writing, but I could have
written it in COBOL.
It was quite straight forward and I consider it a 100% solution to my
problem. It totally eliminated the unnecessary IO.
Another problem that it solved is that now I can declare a file as
fixed record size (normally wo RDW) in MicroFocus COBOL, but maintain
the file on the disk as variable record size (w RDW). We like this
because some of our file manipulation can be generic.
I can not post the code because I was paid to write it. I asked for a
sample from MicroFocus and they quickly replied with sample. I presume
they would extend the same courtesy to licensed user.
--
Michael Potter
|
|
|
|
|