Home > Archive > Unix Programming > June 2004 > data storage
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
|
|
| El Durango 2004-06-25, 7:47 pm |
| Hello I was wondering what is a good, efficient way of storing data that a
server recieves from it's clients. Of course some of the ways to do it is
to write to an XML file or write data to a database. Unfortunately I will
need to run my program on a machine that does not provide a database and has
no XML parsers for C/C++. Is there any better way of storing data other
than writing a flat file w/ delimeters???
| |
| El Durango 2004-06-25, 7:47 pm |
|
Forgot to mention.
Thank you for any advice!
| |
| David Schwartz 2004-06-25, 7:47 pm |
|
"El Durango" <El_Durango@yahoo.com> wrote in message
news:zfuCc.8131$oO4.909@newssvr22.news.prodigy.com...
> Hello I was wondering what is a good, efficient way of storing data that a
> server recieves from it's clients. Of course some of the ways to do it is
> to write to an XML file or write data to a database. Unfortunately I will
> need to run my program on a machine that does not provide a database and
> has
> no XML parsers for C/C++. Is there any better way of storing data other
> than writing a flat file w/ delimeters???
It depends upon what your requirements are. GDBM may be sufficient.
What's the problem with using a flat file with delimeters?
DS
| |
| Jem Berkes 2004-06-25, 7:47 pm |
| > Hello I was wondering what is a good, efficient way of storing data
> that a server recieves from it's clients. Of course some of the ways
> to do it is to write to an XML file or write data to a database.
> Unfortunately I will need to run my program on a machine that does not
> provide a database and has no XML parsers for C/C++. Is there any
> better way of storing data other than writing a flat file w/
> delimeters???
If there is a file system that's friendly to directories containing large
number of files, a nice way to achieve database-like hierarchy without
the overhead (or libraries) of an actual database is to use the file
system as a kind of native database container.
For example, consider a server that's maintaining connections with
several IP address and storing information about each. The directory tree
could look like this:
|-- 10.5.2.2
| |-- cache
| `-- session
|-- 10.5.2.9
|-- 192.168.0.1
`-- 192.168.10.5
|-- cache
`-- session
So in your program, finding data is organized (open $IP/session) and the
structure is inherently meaningful. That is a big plus for maintenance
using external tools, backups, etc. Substantial fault tolerance as well!
This is the kind of approach advocated by Hans Reiser, creator of
ReiserFS. It's different than what we're conventionally used to, but
after months of tests I now share his vision of letting the filesystem do
the organization work, namely using directories. The ReiserFS (available
in the Linux kernel) makes this kind of file storage awfully efficient.
On a volume where files occupy a minimum of 4 KB each, an active database
storing about 30 files/directory in a directory hierarchy showed disk
space usage per file shrink to 0.38 KB (space savings of over 10 times!)
Total number of files was about a quarter-million, most under 500 bytes.
--
Jem Berkes
http://www.sysdesign.ca/
|
|
|
|
|