Home > Archive > VC STL > August 2005 > limitations of fstream to 32 bits values
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
limitations of fstream to 32 bits values
|
|
| PapoudeRepentigny 2005-08-28, 3:57 am |
| Hi all,
I need to access files > 4 GB using C++, so I'm wondering if there's a fix
to VS.NET limitation of 32 bits for the implementation of basic_filebuf.
I found 3 bugs in <iosfw> and <fstream> which prevent STL's way of doing
things, namely define a custom traits class with desired data sizes:
1. in <iosfwd>, in class fpos, a weird thing was written at line 62, where
_Fpos is converted to a long for no apparent reason, preventing the class
from working with 64 bits values;
2 & 3. in <fstream>, basic_filebuf uses fs (FILE*, long, int) instead of
_lsi64(int, _int64, int), again preventing desired behavior when using a
traits class with 64 bits data sizes. Again, this seems weird knowing fpos_t
is defined as _int64, and every other access to the file is compatible with
such a definition.
I've tested an implementation with modified copies of these files, and
everything works fine with huge files, so I'm wondering if anyone is aware of
the problem, and if something is to be done to fix it. Actually, I'm a little
suprised not to find anything nowhere on this topic.
Thanks,
PapoudeRepentigny
| |
| Bo Persson 2005-08-28, 3:57 am |
|
"PapoudeRepentigny" <PapoudeRepentigny@discussions.microsoft.com>
skrev i meddelandet
news:0AE93C99-70DA-40F5-9E7A-ED09EFF3C397@microsoft.com...
> Hi all,
>
> I need to access files > 4 GB using C++, so I'm wondering if there's
> a fix
> to VS.NET limitation of 32 bits for the implementation of
> basic_filebuf.
The limitation isn't on 4GB file size, but on 4GB offsets in file
positioning (s).
>
> I found 3 bugs in <iosfw> and <fstream> which prevent STL's way of
> doing
> things, namely define a custom traits class with desired data sizes:
> 1. in <iosfwd>, in class fpos, a weird thing was written at line 62,
> where
> _Fpos is converted to a long for no apparent reason, preventing the
> class
> from working with 64 bits values;
> 2 & 3. in <fstream>, basic_filebuf uses fs (FILE*, long, int)
> instead of
> _lsi64(int, _int64, int), again preventing desired behavior when
> using a
> traits class with 64 bits data sizes. Again, this seems weird
> knowing fpos_t
> is defined as _int64, and every other access to the file is
> compatible with
> such a definition.
>
They are not bugs in the library, becase it is supposed to work that
way, to be compatible with C file operations.
It could be argued that the real problem is that type long isn't long
enough. Why are both int and long 32 bits, when we obviously need 64
bit values?
> I've tested an implementation with modified copies of these files,
> and
> everything works fine with huge files, so I'm wondering if anyone is
> aware of
> the problem, and if something is to be done to fix it. Actually, I'm
> a little
> suprised not to find anything nowhere on this topic.
Your changes makes the Standard Library a non-standard library, by
using __int64 - a non-standard, non-portable integer type.
Bo Persson
>
> Thanks,
>
> PapoudeRepentigny
| |
| Stephen Howe 2005-08-28, 7:00 pm |
| > I found 3 bugs in <iosfw> and <fstream>...
They are not bugs. They are implementation details.
And I wanted support for 128-bit values, should I describe your solution as
a "bug"?
Please don't abuse the word "bug".
>... which prevent STL's way of doing
> things, namely define a custom traits class with desired data sizes:
> 1. in <iosfwd>, in class fpos, a weird thing was written at line 62, where
> _Fpos is converted to a long for no apparent reason, preventing the class
> from working with 64 bits values;
> 2 & 3. in <fstream>, basic_filebuf uses fs (FILE*, long, int) instead
> of
> _lsi64(int, _int64, int), again preventing desired behavior when using
> a
> traits class with 64 bits data sizes. Again, this seems weird knowing
> fpos_t
> is defined as _int64, and every other access to the file is compatible
> with
> such a definition.
I am not sure I would like the replacement.
fs() might do some bookkeeping behind the scenes in FILE object that
naturally _lsi64() will not.
If you are going replace fs() with _lsi64() you should replace
fgetc(), fputc(), fopen() etc all to be consistent.
> I've tested an implementation with modified copies of these files, and
> everything works fine...
You mean so far you have not noticed a problem.
Does not mean that there is no problem.
Stephen Howe
| |
| paul.derepentigny@gmail.com 2005-08-28, 7:00 pm |
| Hi Stephen,
You're right, it's not a bug, it's an implementation limitation. On my
system I can have fun with huge files using C (not ANSI, I agree), C#,
MFC, the Native API, but not C++.
Other implementations I use on other platforms doesn't have this
limitation, and as I need (and want) to make portable code, I'm a
little bit frustrated on Windows because of this, hence my inapropriate
use of the term bug.
Also, I'm aware that the replacement I suggest is not 'elegant'; it's
not a solution but a way around, a hint these limitations are easily
circumvented.
Regards,
Paul
| |
| Carl Daniel [VC++ MVP] 2005-08-28, 7:00 pm |
| paul.derepentigny@gmail.com wrote:
> Hi Stephen,
>
> You're right, it's not a bug, it's an implementation limitation. On my
> system I can have fun with huge files using C (not ANSI, I agree), C#,
> MFC, the Native API, but not C++.
>
> Other implementations I use on other platforms doesn't have this
> limitation, and as I need (and want) to make portable code, I'm a
> little bit frustrated on Windows because of this, hence my
> inapropriate use of the term bug.
>
> Also, I'm aware that the replacement I suggest is not 'elegant'; it's
> not a solution but a way around, a hint these limitations are easily
> circumvented.
I'd suggest that you enter a suggestion at the MSDN Product Feedback Center:
http://lab.msdn.microsoft.com/productfeedback/
Interestingly, with Visual Studio 2005, if you compile for Win64 then
IOStreams sully supports 64-bit offsets, but if you compile for Win32 it's
the same as VS 2003.
-cd
| |
|
|
|
|
|