Matlab - Do fast hard drives improve matlab performance?

Author

Do fast hard drives improve matlab performance?

Peter Calvert

2005-08-23, 7:03 pm

Has anyone tried to use fast scsi hard drives (15,000 rpm) to improve
the speed of computationally intensive matlab scripts? If so, do they
improve the speed? Thanks!

The Grizzler

2005-08-23, 7:03 pm

Wouldn't memory speed have more influence on speed? Unless of
course you are performing disk access operations frequently. I
don't know for sure but I'd expect CPU/RAM to be most important.

Peter Calvert

2005-08-23, 7:03 pm

The Grizzler wrote:
>
>
> Wouldn't memory speed have more influence on speed? Unless of
> course you are performing disk access operations frequently. I
> don't know for sure but I'd expect CPU/RAM to be most important.
>
>

To some extent that is true but some of the scripts I run are memory
intensive and begin to use large amounts of virtual memory. A
particular bottleneck is the use of the griddata3 function in a step
where I regrid from 3D spherical to 3D Cartesian coordinates, a
memory intensive step that seems unavoidable. It is here that I
wonder if faster disks would be of use.

Any suggestions or constructive comments are much appreciated.

The Grizzler

2005-08-23, 7:03 pm

Well the ultimate way of fighting slower accessed Virtual Memory is
to get more physical ram. But if you can't add more RAM then yes I
agree the faster the hardrive and its interface the better.

John D'Errico

2005-08-23, 7:03 pm

In article <ef11e67.-1@webx.raydaftYaTP>,
"Peter Calvert" <pcalvert@mail.med.upenn.edu> wrote:

> Has anyone tried to use fast scsi hard drives (15,000 rpm) to improve
> the speed of computationally intensive matlab scripts? If so, do they
> improve the speed? Thanks!

If you are relying on a hard drive for speed, then your
code is using too much memory. Its causing Matlab to go
into virtual memory too much. A far better investment
is more RAM. Even better is to invest some time in
optimizing your code.

John

--
The best material model of a cat is another, or
preferably the same, cat.
A. Rosenblueth, Philosophy of Science, 1945

Peter Calvert

2005-08-24, 7:02 pm

Thanks for your input. You are correct to an extent, however
increasing RAM did not help me in this case. I increased my RAM from
2G to 4G with little or no improvement and even with this enormous
amount of RAM have run into "out of memory" errors that were cured
only with increasing the virtual memory paging file size on my hard
disk. I don't believe matlab is set up to utilize such large amounts
of RAM. If you know of a way to improve matlab’s use of RAM I'd
appreciate any suggestions.

In reality the bottleneck in my code is the necessity to regrid from
3D spherical to 3D cartesian coordinates using griddata3, followed by
a 3D convolution. I don't see a way around these steps and, according
to profile, these steps take 95% of the time to run the code. Perhaps
you know of a way to speed up griddata3?

Thanks – Peter

John D'Errico wrote:
>
>
> In article <ef11e67.-1@webx.raydaftYaTP>,
> "Peter Calvert" <pcalvert@mail.med.upenn.edu> wrote:
>
> improve
do[color=darkred]
> they
>
> If you are relying on a hard drive for speed, then your
> code is using too much memory. Its causing Matlab to go
> into virtual memory too much. A far better investment
> is more RAM. Even better is to invest some time in
> optimizing your code.
>
> John
>
>
> --
> The best material model of a cat is another, or
> preferably the same, cat.
> A. Rosenblueth, Philosophy of Science, 1945
>

Steve Amphlett

2005-08-24, 7:02 pm

<snip, fast hard drives...

I find a fast, hard drive can clear the mind sometimes. But you
can't beat a slow, pensive bicycle ride home if you really want to
improve Matlab performance.

Dan Hensley

2005-08-24, 7:02 pm

On Wed, 24 Aug 2005 09:18:28 -0400, Peter Calvert wrote:

> Thanks for your input. You are correct to an extent, however
> increasing RAM did not help me in this case. I increased my RAM from
> 2G to 4G with little or no improvement and even with this enormous
> amount of RAM have run into "out of memory" errors that were cured
> only with increasing the virtual memory paging file size on my hard
> disk. I don't believe matlab is set up to utilize such large amounts
> of RAM. If you know of a way to improve matlab’s use of RAM I'd
> appreciate any suggestions.

You can switch to the 64-bit Matlab running on AMD64 on Linux. Sounds
like you're running into 32-bit addressing limitations, so 4G RAM isn't
going to help you. You didn't specify what OS you're using. If you're
using Windows, the maximum amount of memory you can use in a session is a
bit less than 1.5G.

Dan

[color=darkred]
>
> In reality the bottleneck in my code is the necessity to regrid from
> 3D spherical to 3D cartesian coordinates using griddata3, followed by
> a 3D convolution. I don't see a way around these steps and, according
> to profile, these steps take 95% of the time to run the code. Perhaps
> you know of a way to speed up griddata3?
>
> Thanks – Peter
>
> John D'Errico wrote:
> do

Peter Boettcher

2005-08-24, 7:02 pm

"Peter Calvert" <pcalvert@mail.med.upenn.edu> writes:

> Thanks for your input. You are correct to an extent, however
> increasing RAM did not help me in this case. I increased my RAM from
> 2G to 4G with little or no improvement and even with this enormous
> amount of RAM have run into "out of memory" errors that were cured
> only with increasing the virtual memory paging file size on my hard
> disk. I don't believe matlab is set up to utilize such large amounts
> of RAM. If you know of a way to improve matlab's use of RAM I'd
> appreciate any suggestions.
>
> In reality the bottleneck in my code is the necessity to regrid from
> 3D spherical to 3D cartesian coordinates using griddata3, followed by
> a 3D convolution. I don't see a way around these steps and, according
> to profile, these steps take 95% of the time to run the code. Perhaps
> you know of a way to speed up griddata3?

Perhaps you could recode your algorithm to use interp3 instead of
griddata3. Since you already have regularly spaced coordinates,
interpolation would be much faster. Be sure each of the spherical
coordinates are specified monotonically (0 to 360). Then generate
your desired cartesian grid using meshgrid, and reverse-convert those
points into spherical coordinates. Interpolate the data values at
those converted points, and you should be all set.

If this won't work for you, try recoding to use griddata3 on blocks of
the data, and assemble them later. Maybe regrid each of the 8
quadrants (octants?) independently. You could use blocks that overlap
each other slightly, and trim the outputs to the true size, in order
to help the seam problem.

--
Peter Boettcher <boettcher@ll.mit.edu>
MIT Lincoln Laboratory
MATLAB FAQ: http://www.mit.edu/~pwb/cssm/

Peter Calvert

2005-08-24, 7:02 pm

Peter Boettcher wrote:
>
>
> "Peter Calvert" <pcalvert@mail.med.upenn.edu> writes:
>
> from
> enormous
> cured
> hard
> amounts
> from
> followed by
> according
> Perhaps
>
> Perhaps you could recode your algorithm to use interp3 instead of
> griddata3. Since you already have regularly spaced coordinates,
> interpolation would be much faster. Be sure each of the spherical
> coordinates are specified monotonically (0 to 360). Then generate
> your desired cartesian grid using meshgrid, and reverse-convert
> those
> points into spherical coordinates. Interpolate the data values at
> those converted points, and you should be all set.
>
> If this won't work for you, try recoding to use griddata3 on blocks
> of
> the data, and assemble them later. Maybe regrid each of the 8
> quadrants (octants?) independently. You could use blocks that
> overlap
> each other slightly, and trim the outputs to the true size, in
> order
> to help the seam problem.
>
>
> --
> Peter Boettcher <boettcher@ll.mit.edu>
> MIT Lincoln Laboratory
> MATLAB FAQ: <http://www.mit.edu/~pwb/cssm/>
>

Thanks a lot. I'll try your approach.

Peter Calvert

2005-08-30, 7:04 pm

Rune Allnor

2005-08-31, 3:59 am

Steve Amphlett wrote:
> <snip, fast hard drives...
>
> I find a fast, hard drive can clear the mind sometimes. But you
> can't beat a slow, pensive bicycle ride home if you really want to
> improve Matlab performance.

Hmmm... too much distractions by keeping your balance and
staying on the road.

When I worked on my PhD thesis I walked every day, 45 minutes
to one hour each way, to and from work. Those couple of hours
were utterly essential to get anything done. I guess they were
my "disk defragmenting time", to keep it in the computer
analogy. It's interesting -- spooky, even -- how the best
ideas suddenly appear when one thinks of absolutely nothing.

Using a bicycle, let alone driving a car, is enough 'brain work'
to block those ideas.

Rune

Peter Boettcher

2005-08-31, 7:01 pm

"Peter Calvert" <pcalvert@mail.med.upenn.edu> writes:

> I tried the approaches you suggested. I could not use interp3 because
> it assumes a uniform Cartesian grid, which is obviously not possible
> in spherical coordinates. But your idea of taking advantage of
> symmetry and applying griddata3 to a single octant was very useful,
> speeding up the processing about eightfold, as expected.
> Interestingly we have used this strategy before - but I had forgotten
> about it. Thanks for the reminder!!

I still think interp3 should work. It doesn't require a uniform grid,
just a monotonic and plaid grid. I assume your thetas run (or can
run) from 0 to 360 degrees, and your phis from -90 to 90, and your R's
from 0 to whatever, and you have a sample for each theta,phi,R.
That's monotonic and plaid!

The points you want to sample at are not, but that's OK, that's the
point of interp3. They're uniform on a cartesian grid, and
reverse-converted to spherical, wherever they may fall.

In 2D:

[theta r] = meshgrid(linspace(-pi,pi,1000), linspace(0,10,1000));
% These don't even have to be linear!
s = 5-sqrt(r); % Polar coordinate surface, radially symmetric

% Generate a cartesian grid in the same area
x1d = linspace(-0.5,0.5,250);
y1d = linspace(-0.5,0.5,250);
[x,y]=meshgrid(x1d, y1d);

% Generate the polar equivalents of those cartesian points
r_cart = sqrt(x.^2 + y.^2);
theta_cart = atan2(y,x);

% Interp the polar surface to the new cartesian points
scart = interp2(theta, r, s, theta_cart, r_cart);

% Plot it up!
imagesc(x1d,y1d,scart);

--
Peter Boettcher <boettcher@ll.mit.edu>
MIT Lincoln Laboratory
MATLAB FAQ: http://www.mit.edu/~pwb/cssm/

Sponsored Links