Home > Archive > Clipper > October 2007 > Clipper vs xHarbour Benchmark Tests
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Clipper vs xHarbour Benchmark Tests
|
|
|
| Well, excuse me. I had previously done a timing test between Clipper
and xHarbour about a month ago and the result was xHarbour was about
20% slower (or so I thought). I tried to locate the tests, but it is
either in the bit bucket or alzheimer's creeping in. So, I tried the
timing test on a current project: 1 million records, each s ing into
another dbf, then looping through a small 300 record dbf, then
outputting 1+ million records with additional fields. Here are the
unscientific results. Clipper using DBFCDX 41.67 minutes. xHarbour
using DBFCDX 33.48 minutes, xHarbour DBFNTX 31.21 minutes. In this
test, xHarbour was significantly FASTER. Seems suspicious the NTX was
a little faster than CDX, but I'm not going to argue. I guess I
should have rebooted before each run. I may do the test again when I
have more time. Any comments on the times above? Are they consistent
or inconsistent with anyone else's tests? By the way, the tests were
on W2000, AMD 2.2GHz, SCSI 160 drive, Clipper 5.3b, and the latest
xHarbour. Might be interesting to run these tests on my XP computer.
I'll work on that.
Thanks for reading,
Ted
| |
| N:dlzc D:aol T:com \(dlzc\) 2007-09-28, 3:55 am |
| Dear Ted:
"Ted" <ted.kobayashi@gmail.com> wrote in message
news:1190948453.740083.48980@k79g2000hse.googlegroups.com...
> Well, excuse me. I had previously done a timing
> test between Clipper and xHarbour about a month
> ago and the result was xHarbour was about 20%
> slower (or so I thought). I tried to locate the tests,
> but it is either in the bit bucket or alzheimer's
> creeping in.
What? ;> )
> So, I tried the timing test on a current project: 1
> million records, each sing into another dbf, then
> looping through a small 300 record dbf, then
> outputting 1+ million records with additional fields.
> Here are the unscientific results. Clipper using
> DBFCDX 41.67 minutes. xHarbour using DBFCDX
> 33.48 minutes, xHarbour DBFNTX 31.21 minutes.
> In this test, xHarbour was significantly FASTER.
> Seems suspicious the NTX was a little faster than
> CDX, but I'm not going to argue. I guess I should
> have rebooted before each run.
That only starts the caches clean, but yes.
> I may do the test again when I have more time.
> Any comments on the times above? Are they
> consistent or inconsistent with anyone else's
> tests? By the way, the tests were on W2000,
> AMD 2.2GHz, SCSI 160 drive, Clipper 5.3b, and
> the latest xHarbour. Might be interesting to run
> these tests on my XP computer. I'll work on that.
I'd guess that you won't see a major difference on XP. Where I
think you will get your reverse result is on Win98 or WinMe,
where you have a DOS foundation, instead of something spliced
onto WinNT that "acts" like DOS. Also the underlying file system
(FAT vs. FAT32 vs. NTFS) can be a major contributor.
Additionally, if you incorporate one of the "visual" libraries
for whatever reason, this overhead can also swallow some CPU...
slowing the process down.
David A. Smith
| |
| Massimo Belgrano 2007-09-28, 3:55 am |
| On 28 Set, 05:00, Ted <ted.kobaya...@gmail.com> wrote:
> Well, excuse me. I had previously done a timing test between Clipper
> and xHarbour about a month ago and the result was xHarbour was about
> 20% slower (or so I thought). I tried to locate the tests, but it is
> either in the bit bucket or alzheimer's creeping in. So, I tried the
> timing test on a current project: 1 million records, each sing into
> another dbf, then looping through a small 300 record dbf, then
> outputting 1+ million records with additional fields. Here are the
> unscientific results. Clipper using DBFCDX 41.67 minutes. xHarbour
> using DBFCDX 33.48 minutes, xHarbour DBFNTX 31.21 minutes. In this
> test, xHarbour was significantly FASTER. Seems suspicious the NTX was
> a little faster than CDX, but I'm not going to argue. I guess I
> should have rebooted before each run. I may do the test again when I
> have more time. Any comments on the times above? Are they consistent
> or inconsistent with anyone else's tests? By the way, the tests were
> on W2000, AMD 2.2GHz, SCSI 160 drive, Clipper 5.3b, and the latest
> xHarbour. Might be interesting to run these tests on my XP computer.
> I'll work on that.
>
> Thanks for reading,
>
> Ted
Can post your prg who made this test?
| |
| Mike Evans 2007-09-28, 7:55 am |
| On Thu, 27 Sep 2007 20:00:53 -0700, Ted wrote:
> Well, excuse me. I had previously done a timing test between Clipper
> and xHarbour about a month ago and the result was xHarbour was about
> 20% slower (or so I thought). I tried to locate the tests, but it is
> either in the bit bucket or alzheimer's creeping in. So, I tried the
> timing test on a current project: 1 million records, each sing into
> another dbf, then looping through a small 300 record dbf, then
> outputting 1+ million records with additional fields. Here are the
> unscientific results. Clipper using DBFCDX 41.67 minutes. xHarbour
> using DBFCDX 33.48 minutes, xHarbour DBFNTX 31.21 minutes. In this
> test, xHarbour was significantly FASTER. Seems suspicious the NTX was
> a little faster than CDX, but I'm not going to argue. I guess I
> should have rebooted before each run. I may do the test again when I
> have more time. Any comments on the times above? Are they consistent
> or inconsistent with anyone else's tests? By the way, the tests were
> on W2000, AMD 2.2GHz, SCSI 160 drive, Clipper 5.3b, and the latest
> xHarbour. Might be interesting to run these tests on my XP computer.
> I'll work on that.
>
> Thanks for reading,
>
> Ted
Ted,
I've made a lot of bebchmark tests for xharbour VS clipper 5.3. In general
xHarbour is faster except from the following situations:
Screen display. xHarbour is slower (dependant from the GT) and its logical
because its display in screen throught windows api and to overcome this you
must be carefull if you display continiously many thinks.
Ordkeyno(), Ordkeycount(). They are a lot slower in xHarbour because
xharbour does not cache them.
Optimized filters (RMDBFCDX) also under some circumstances are a lot slower
again because xHarbour does not cache database files. For the last 2
Przemek said that maybee some day he will add caching in dbf (as clipper).
For the first i think that Przemek also made some modifications but only
for Linux Gts to speed up such operations.
Also if you use RMDBFCDX and optimized filters you must know that the
current RMDBFCDX does not optimize filter without the second part of
logical expressions ....fielda == .T.. This is allready fixed and it will
be included in the next version.
Regards
Mike Evans
| |
| Claudio Voskian 2007-09-28, 6:55 pm |
| Hi Ted and others
"Ted" <ted.kobayashi@gmail.com> escribió en el mensaje
news:1190948453.740083.48980@k79g2000hse.googlegroups.com...
> Well, excuse me. I had previously done a timing test between Clipper
> and xHarbour about a month ago and the result was xHarbour was about
> 20% slower (or so I thought). I tried to locate the tests, but it is
> either in the bit bucket or alzheimer's creeping in. So, I tried the
> timing test on a current project: 1 million records, each sing into
> another dbf, then looping through a small 300 record dbf, then
> outputting 1+ million records with additional fields. Here are the
> unscientific results. Clipper using DBFCDX 41.67 minutes. xHarbour
> using DBFCDX 33.48 minutes, xHarbour DBFNTX 31.21 minutes. In this
> test, xHarbour was significantly FASTER. Seems suspicious the NTX was
> a little faster than CDX, but I'm not going to argue. I guess I
> should have rebooted before each run. I may do the test again when I
> have more time. Any comments on the times above? Are they consistent
> or inconsistent with anyone else's tests? By the way, the tests were
> on W2000, AMD 2.2GHz, SCSI 160 drive, Clipper 5.3b, and the latest
> xHarbour. Might be interesting to run these tests on my XP computer.
> I'll work on that.
>
> Thanks for reading,
>
> Ted
>
I'm using the pro version and found that it is very slow when used in a
network environment.
Every networks is different, I know, and I don't know if the one I use is
properly adjusted (it is my client's network, where the real things are
going, which, by the way, is a novell netware 5.1 in a P4 server, tcp/ip +
ipx, 10/100 switched network, workstations are w2k pro).
In fact, anything I tried to do was much slower compared with the equivalent
clipper application +comix or +six3 still in use (up to now, they were just
recompiled and do minimal changes to make it work).
Don't know how to get a quantum in speed ... it was one of the reasons to go
to xharbour (between others, of course).
Examples:
build .cdx indexes from scratch (in big databases of >500.000 records),
slower than cl5.2e
'select' a set of records using bit mapped filters (like the ones in comix
or six3), painly slow
browse a table with or without filter installed, a bit slow than in clipper
BUT: If it runs locally (copying all the files to C:\, so no network is
involved), it works really fast and works great (it is a cache related
thing).
My next try will be a migration to mysql, but it is in my mind only.
Have a nice day,
--
Claudio Voskian
ICQ#: 18122595 - Aol/Msn/Skype/Y!: cvoskian
Buenos Aires - Argentina
| |
| Mike Evans 2007-09-28, 6:55 pm |
| On Fri, 28 Sep 2007 10:42:25 -0300, Claudio Voskian wrote:
> Hi Ted and others
>
> "Ted" <ted.kobayashi@gmail.com> escribió en el mensaje
> news:1190948453.740083.48980@k79g2000hse.googlegroups.com...
>
> I'm using the pro version and found that it is very slow when used in a
> network environment.
> Every networks is different, I know, and I don't know if the one I use is
> properly adjusted (it is my client's network, where the real things are
> going, which, by the way, is a novell netware 5.1 in a P4 server, tcp/ip +
> ipx, 10/100 switched network, workstations are w2k pro).
>
> In fact, anything I tried to do was much slower compared with the equivalent
> clipper application +comix or +six3 still in use (up to now, they were just
> recompiled and do minimal changes to make it work).
>
> Don't know how to get a quantum in speed ... it was one of the reasons to go
> to xharbour (between others, of course).
>
> Examples:
> build .cdx indexes from scratch (in big databases of >500.000 records),
> slower than cl5.2e
> 'select' a set of records using bit mapped filters (like the ones in comix
> or six3), painly slow
> browse a table with or without filter installed, a bit slow than in clipper
>
> BUT: If it runs locally (copying all the files to C:\, so no network is
> involved), it works really fast and works great (it is a cache related
> thing).
>
> My next try will be a migration to mysql, but it is in my mind only.
>
> Have a nice day,
Claudio,
One user access the file from the network drive or many concurrent?
Also which xHarbour RDD you are using and with what DBFLOCKSCHEME value?
Regards
Mike Evans
| |
| Claudio Voskian 2007-09-28, 6:55 pm |
| Mike
"Mike Evans" <makis1970@hotmail.com> escribió en el mensaje
news:1q8f6citbglnt$.sc3wi32oydfk.dlg@40tude.net...
> On Fri, 28 Sep 2007 10:42:25 -0300, Claudio Voskian wrote:
>
>
> Claudio,
> One user access the file from the network drive or many concurrent?
> Also which xHarbour RDD you are using and with what DBFLOCKSCHEME value?
> Regards
> Mike Evans
The usual thing in a network environment is that there are many users
accessing files concurrently.
Except when indexing (i.e., "exclusive use" by the nature of the process),
the remaining tests were done with shared access.
The dbflockscheme I use is 3 (don't know what could it affect), and the rdd
is rmdbfcdx (in the app. that uses six3, also uses sde61.lib); take into
account that for this tests the files are not being shared between the
clipper version and the xharbour one. They are installed on different
folders on the same file server, but not shared at all between them (this is
just to avoid further problems).
The test was done in a normal labor day (yesterday), with all people working
so the server and the network were busy (as usual). Obviously my customer is
a little worried.
Regards,
--
Claudio Voskian
ICQ#: 18122595 - Aol/Msn/Skype/Y!: cvoskian
Buenos Aires - Argentina
| |
| Patrick Mast 2007-09-30, 6:55 pm |
| Hello Mike,
> Optimized filters (RMDBFCDX) also under some circumstances are a lot slower
> again because xHarbour does not cache database files. For the last 2
> Przemek said that maybee some day he will add caching in dbf (as clipper).
I have a big application that was compiled with
Clipper+FiveWin+SIx3+MachSix. Since I compile with xHarbour
Builder+FWH+RMDBFCDX my users only feedback is that my app runs faster
now.
Just my experience. :)
--
Sincerely,
Patrick Mast,
xHarbour.com Inc.
http://www.xHarbour.com
| |
| Scott Burke 2007-10-01, 6:55 pm |
| When you delete a record you have to rebuild all indexes that goes with that
DBF.
That can be a time consuming thing. Add your network and It sound like you
are
having a big timing problem.
Consider this:
Use a delete field. When some one deletes a record what they are really
doing is
setting a flag to 'TRUE'. Of course you will have to change all programs to
ignore those
records who delete field = true. This method should be fast and cut out
your timing
problems. The Delete field should not be in any index!
Then every night the Administator ( or someone ) can run a program that
deletes all
indexes then deletes all records who delete field = true and then rebuild
the indexes.
Since indexes are easly damaged this method can save your self a lot of
problems by doing this.
Your DBF's will be smaller and faster.
Scott Burke
Just my 2 cents.
"Mike Evans" <makis1970@hotmail.com> wrote in message
news:1q8f6citbglnt$.sc3wi32oydfk.dlg@40tude.net...
> On Fri, 28 Sep 2007 10:42:25 -0300, Claudio Voskian wrote:
>
>
> Claudio,
> One user access the file from the network drive or many concurrent?
> Also which xHarbour RDD you are using and with what DBFLOCKSCHEME value?
> Regards
> Mike Evans
| |
| Jerzy Mi¶ 2007-10-02, 7:55 am |
| > When you delete a record you have to rebuild all indexes that goes with
> that DBF.
Arte you sure ?
| |
| Patrick Mast 2007-10-02, 7:55 am |
| Hello Jerry,
>
> Arte you sure ?
No, its not true. Maybe he wants to say taht when you delete a record
the index files wil be 'thought' by xHarbour.
--
Sincerely,
Patrick Mast,
xHarbour.com Inc.
http://www.xHarbour.com
| |
| Mike Evans 2007-10-02, 7:55 am |
| On Fri, 28 Sep 2007 12:43:03 -0300, Claudio Voskian wrote:
> Mike
>
> "Mike Evans" <makis1970@hotmail.com> escribió en el mensaje
> news:1q8f6citbglnt$.sc3wi32oydfk.dlg@40tude.net...
>
> The usual thing in a network environment is that there are many users
> accessing files concurrently.
> Except when indexing (i.e., "exclusive use" by the nature of the process),
> the remaining tests were done with shared access.
>
> The dbflockscheme I use is 3 (don't know what could it affect), and the rdd
> is rmdbfcdx (in the app. that uses six3, also uses sde61.lib); take into
> account that for this tests the files are not being shared between the
> clipper version and the xharbour one. They are installed on different
> folders on the same file server, but not shared at all between them (this is
> just to avoid further problems).
>
> The test was done in a normal labor day (yesterday), with all people working
> so the server and the network were busy (as usual). Obviously my customer is
> a little worried.
>
> Regards,
Claudio,
Try DBFLOCKSCHEME 2. (Its like clipper 5.3 default sharing locking scheme.
Also As i said until the latest version (i didn't try it until now) filters
need the second part for logical expressions ( fielda == .t. ) to be
optimized. Clipper does need this. Also DEPENTED FROM THE FILE SIZE and
because clipper using cache in dbf files is faster tha xHarbour (this one
is aknoeledge by przemek). In general as Przemek said when the file getting
bigger (and the clipper caching mechanism is not possible to cache)
xHarbour again getting faster). Also especially for the Browse look
carefully for calls to ordkeycoynt ordkeyno. This 2 fuctions are a lot
slower in xHarbour RMDBFCDX because xHarbour does not use cache for this
operations. I hope that someday Przemek will add Caching to RMDBFCDX and
i'm sure that will see xHarbour shine under all circumstances. Also have in
mind the sreen display under xHarbour is a lot slower than clipper and from
GT to Gt is even worst. Fo such operations that display continiously in the
screen add special algorithm no to display allways in the screen.
| |
| Claudio Voskian 2007-10-02, 6:55 pm |
| Mike
"Mike Evans" <makis1970@hotmail.com> escribió en el mensaje
news:4701f383$0$1346$834e42db@reader.greatnowhere.com...
> On Fri, 28 Sep 2007 12:43:03 -0300, Claudio Voskian wrote:
>
>
> Claudio,
> Try DBFLOCKSCHEME 2. (Its like clipper 5.3 default sharing locking scheme.
> Also As i said until the latest version (i didn't try it until now)
> filters
> need the second part for logical expressions ( fielda == .t. ) to be
> optimized. Clipper does need this. Also DEPENTED FROM THE FILE SIZE and
> because clipper using cache in dbf files is faster tha xHarbour (this one
> is aknoeledge by przemek). In general as Przemek said when the file
> getting
> bigger (and the clipper caching mechanism is not possible to cache)
> xHarbour again getting faster). Also especially for the Browse look
> carefully for calls to ordkeycoynt ordkeyno. This 2 fuctions are a lot
> slower in xHarbour RMDBFCDX because xHarbour does not use cache for this
> operations. I hope that someday Przemek will add Caching to RMDBFCDX and
> i'm sure that will see xHarbour shine under all circumstances. Also have
> in
> mind the sreen display under xHarbour is a lot slower than clipper and
> from
> GT to Gt is even worst. Fo such operations that display continiously in
> the
> screen add special algorithm no to display allways in the screen.
---
Thank you for responding me.
Will try dbflockscheme=2, but I think its functionality is related with the
way (or the offset at which) the locks are set in a database; it is a
compatibility function when sharing the same set of databases using
different applications (one written in clipper, another with xh). Don't
think it could affect performance, but I'm not an expert in that.
Also I will look at the expressions used in the calls to the filtering
functions (with rmdbfcdx), some of they are 'pre-programmed' in the source
files, but the user can add its own expressions (I left them instructions in
how-to-do that).
Anyway, I should have to experiment a lot with this new toy before going to
some flavour of sql in the near future.
For the screen refresh been slow, it is not as important anyway. Is there
any replacements for the functions you mentioned (ordkeycount / ordkeyno)?
An out of topic:
Too much people were talking about Przemyslaw Czerpak as a genius; I
understand that he has been one of the main developers of the rdd layers...
so, he should be a genius; my questions are:
- What is he doing now?
- Does he 'work' with/at the xharbour team?
- Can be contacted in some way?
I see in the news that he actively responded to any question in the past,
but I don't see any post from him from long time ago.
Have a nice day, it's rainy here.
Claudio
| |
| Massimo Belgrano 2007-10-02, 6:55 pm |
| On 2 Ott, 09:27, Mike Evans <makis1...@hotmail.com> wrote:
> On Fri, 28 Sep 2007 12:43:03 -0300, Claudio Voskian wrote:
>
>
>
to[color=darkred]
as[color=darkred]
nt[color=darkred]
>
>
>
a[color=darkred]
e is[color=darkred]
re[color=darkred]
/ip[color=darkred]
>
>
s to[color=darkred]
>
),[color=darkred]
>
is[color=darkred]
>
>
>
e?[color=darkred]
>
s),[color=darkred]
>
rdd[color=darkred]
is is[color=darkred]
>
rking[color=darkred]
er is[color=darkred]
>
>
> Claudio,
> Try DBFLOCKSCHEME 2. (Its like clipper 5.3 default sharing locking scheme.
> Also As i said until the latest version (i didn't try it until now) filte=
rs
> need the second part for logical expressions ( fielda =3D=3D .t. ) to be
> optimized. Clipper does need this. Also DEPENTED FROM THE FILE SIZE and
> because clipper using cache in dbf files is faster tha xHarbour (this one
> is aknoeledge by przemek). In general as Przemek said when the file getti=
ng
> bigger (and the clipper caching mechanism is not possible to cache)
> xHarbour again getting faster). Also especially for the Browse look
> carefully for calls to ordkeycoynt ordkeyno. This 2 fuctions are a lot
> slower in xHarbour RMDBFCDX because xHarbour does not use cache for this
> operations. I hope that someday Przemek will add Caching to RMDBFCDX and
> i'm sure that will see xHarbour shine under all circumstances. Also have =
in
> mind the sreen display under xHarbour is a lot slower than clipper and fr=
om
> GT to Gt is even worst. Fo such operations that display continiously in t=
he
> screen add special algorithm no to display allways in the screen.- Nascon=
di testo tra virgolette -
>
> - Mostra testo tra virgolette -
Wich is the best for adantage database (cdx) coesistence?
0 Default locking scheme
1 Clipper 5.2 locking scheme
2 Clipper 5.3 locking scheme
3 Visual FoxPro locking scheme
4 Emulated shared locking
5 Locking scheme for files > 4GB
| |
| Marek Horodyski 2007-10-02, 6:55 pm |
|
U¿ytkownik "Claudio Voskian" <mixxell@hotmail.com> napisa³ w wiadomoœci
news:470247e0$0$1341$834e42db@reader.greatnowhere.com...
> Mike
[ ... ]
> so, he should be a genius; my questions are:
> - What is he doing now?
Przemek now is hard working about Harbour.
You can see on www.harbour-project.org .
> - Does he 'work' with/at the xharbour team?
Ocasionally.
Regards,
Marek Horodyski
| |
|
|
|
|
| Massimo Belgrano 2007-10-02, 6:55 pm |
| On 2 Ott, 23:47, Massimo Belgrano <massimo.belgr...@gmail.com> wrote:
> On 2 Ott, 22:40, "Enrico Maria Giordano"
>
>
>
>
>
> <e.m.giord...@emagsoftware.it> wrote:
>
>
>
>
>
and here the mailing where genius works
http://lists.harbour-project.org/pipermail/harbour/
| |
| Marcelo Sturm 2007-10-05, 6:55 pm |
| Why Przemyslaw Czerpak change xharbour for harbour?
[ ]s, Marcelo Sturm!
| |
| Massimo Belgrano 2007-10-08, 6:55 pm |
| it have a little different idea about future of product and think that in
harbour is more simple giving his direction
"Marcelo Sturm" <marcelo.sturm@gmail.com> ha scritto nel messaggio
news:47066d9b$0$1346$834e42db@reader.greatnowhere.com...
> Why Przemyslaw Czerpak change xharbour for harbour?
>
> [ ]s, Marcelo Sturm!
|
|
|
|
|