Big tables

I maintain a fairly elaborate set of .dbf tables representing images of
tables in an Informix engine on a Sun server. My images are created and
maintained on a Windows 2k server on a daily (overnight) basis.

I create the .dbf with a program of my own which reads a text file from
the Informix server and builds a .dbf on the windows side. It is
essentially 'append from' and very fast.

There are 100 or so of these .dbf tables created every morning (M..F)
and some of them get larger on a daily basis.

There is some preprocessing of these tables by Clipper apps I have
written. One day last month one of the Clipper apps failed. It had run
flawlessly five days a w for five years. Now it fails..

A little troubleshooting suggested it failed while attempting to index
the principal .dbf table. I noted that the table size was approaching 2
Gigabytes. Clipper didn't seem to have any problem with the size of the
table so I determined to 'cook' it down some before indexing it.

I got the file size down to 900 Megabytes or so and indexing it went
smoothly. Problem solved.

Maybe not. During the day we use Visual FoxPro 6.0 to do various
interesting things using my .dbf images. Yesterday, Feb 28, VFP6.0
attempted to open my 2GB table, barfed and declared "Not a table".

Clipper thinks it's a table but can't index it.

Underlying filesystem is NTFS. Is there a known way through or around
this 2GB file limit?

--
Joe Wright
"Everything should be made as simple as possible, but not simpler."
--- Albert Einstein ---

Dear Joe Wright:

"Joe Wright" <joewwright@comcast.net> wrote in message
 news:tKqdnRe9vJQTJlXanZ2dnUVZ_v6rnZ2d@co
mcast.com...
> I maintain a fairly elaborate set of .dbf tables
> representing images of tables in an Informix
> engine on a Sun server. My images are created
> and maintained on a Windows 2k server on a
> daily (overnight) basis.
>
> I create the .dbf with a program of my own which
> reads a text file from the Informix server and
> builds a .dbf on the windows side. It is essentially
> 'append from' and very fast.

What RDD?

> There are 100 or so of these .dbf tables created
> every morning (M..F) and some of them get
> larger on a daily basis.
>
> There is some preprocessing of these tables by
> Clipper apps I have written. One day last month
> one of the Clipper apps failed. It had run flawlessly five days
> a w for five years. Now it
> fails..
>
> A little troubleshooting suggested it failed while
> attempting to index the principal .dbf table.

How many keys, and what length?

> I noted that the table size was approaching 2 Gigabytes.
> Clipper didn't seem to have any problem
> with the size of the table so I determined to 'cook' it
> down some before indexing it.
>
> I got the file size down to 900 Megabytes or so and
> indexing it went smoothly. Problem solved.
>
> Maybe not. During the day we use Visual FoxPro
> 6.0 to do various interesting things using my .dbf
> images. Yesterday, Feb 28, VFP6.0 attempted to
> open my 2GB table, barfed and declared "Not a
> table".
>
> Clipper thinks it's a table but can't index it.
>
> Underlying filesystem is NTFS. Is there a known way
> through or around this 2GB file limit?

Have a sense of humor, I am not trying to be a smart-a**:
- xHarbour has no such low limitations, and will accept your
Clipper code... depending on your 3rd party libs.
- xHarbour has an SQL RDD available, so that you could directly
access the server data, and let it do all the work.
- you might find some code here that will let Clipper (but
certainly xHarbour) access the data...
http://www.iiug.org/software/software_index.html
... maybe via OLE.

As to Visual FauxPro, I believe Micro$haft is making that open
source, and it is certainly higher than 6.0 now.

xHarbour has its own newsgroup on comp.lang.xharbour.
www.xharbour.org for the free version
www.xharbour.com for the commerical version, and the SQL RDD.

David A. Smith

Joe

> Underlying filesystem is NTFS. Is there a known way through or around this 2GB fil
e limit?

Advantage DBS (IIRC seeing that it supports upto 4G)
or
Move the app to a 32bit system and/or move from dbf to a sql rmdbs
eg. Harbour, I believe can go to the terabyte range

You might also think about splitting the files involved.

--
CYA
Steve

N:dlzc D:aol T:com (dlzc) wrote:
> Dear Joe Wright:
>
> "Joe Wright" <joewwright@comcast.net> wrote in message
>  news:tKqdnRe9vJQTJlXanZ2dnUVZ_v6rnZ2d@co
mcast.com... 
>
> What RDD?
>
None. It's a tight little C program that takes Informix pipe delimited
files and 'appends' the lines to a .dbf table.
 
>
> How many keys, and what length?
>
I can't say exactly as I'm at home now, not at work. Certainly key
length is very important.
 
>
> Have a sense of humor, I am not trying to be a smart-a**:
> - xHarbour has no such low limitations, and will accept your
> Clipper code... depending on your 3rd party libs.
> - xHarbour has an SQL RDD available, so that you could directly
> access the server data, and let it do all the work.
> - you might find some code here that will let Clipper (but
> certainly xHarbour) access the data...
> http://www.iiug.org/software/software_index.html
> ... maybe via OLE.
>
A sense of humor? Have you heard the one about the ... ?
We don't send donkeys to school because nobody likes a smart ass. :-)
I will again look into xHarbour and SQLRDD. Thanks.

> As to Visual FauxPro, I believe Micro$haft is making that open
> source, and it is certainly higher than 6.0 now.
>
> xHarbour has its own newsgroup on comp.lang.xharbour.
> www.xharbour.org for the free version
> www.xharbour.com for the commerical version, and the SQL RDD.
>
> David A. Smith
>
>
Thanks David.

--
Joe Wright
"Everything should be made as simple as possible, but not simpler."
--- Albert Einstein ---

Stephen Quinn wrote:
> Joe
> 
>
> Advantage DBS (IIRC seeing that it supports upto 4G)
> or
> Move the app to a 32bit system and/or move from dbf to a sql rmdbs
> eg. Harbour, I believe can go to the terabyte range
>
> You might also think about splitting the files involved.
>
Advantage DBS might not be an option here. Normalizing (sp?) to smaller
tables and relating them might well be the answer. Thank you.

--
Joe Wright
"Everything should be made as simple as possible, but not simpler."
--- Albert Einstein ---

AMATEUR P0RN
http://amazing-cumshot-movies.com/watch?v=218571

Cumshot, facial, hardcore! Fast download speed
[IMG]http://amazing-cumshot-movies.com/
shot3.jpg[/IMG]