Matlab - How to remnove an annoying bottleneck

Author

How to remnove an annoying bottleneck

Paul McGuire

2007-06-28, 8:12 am

Hi all, I've got this piece of code in a little program of mine:

%loop filenames and do calculation
for j=1:numfiles
x=imread(strcat(direc,e{j},'.jpg'),'jpg');
LAB = applycform(x,M);
n =LAB(:,:,2);
booln = uint8(n~=128);
m{1,j} = sum(booln .* n,1) ./ sum(booln,1);
end

In a nutshell I have a series of images, this opens each one, applies
the colour transformation, and finds the means of the columns in
(:,:,2), with a bit of code so the averaging ignores the whitespace.

Trouble is this is taking in the order of 5-10 minutes depending on
how many image files I have to loop through.

What I'd prefer to do is open the individual images, concatenate them
first, then carry out the colour transformation, which I think would
be a lot quicker.

I was attempting to do this but ran into the problem that the image
files are various sizes, so I thought of padding them all out to the
same size, but creating 'padding' of the right dimensions then
applying it to each image prior to concatenating was making the
program just as slow.

Does anyone have any suggestions as to how I might go about improving
this piece of code in any way shape or form?

snewdl@gmail.com

2007-06-28, 7:17 pm

On Jun 28, 4:21 am, "Paul McGuire" <mcgui...@hotmail.com> wrote:
> Hi all, I've got this piece of code in a little program of mine:
>
> %loop filenames and do calculation
> for j=1:numfiles
> x=imread(strcat(direc,e{j},'.jpg'),'jpg');
> LAB = applycform(x,M);
> n =LAB(:,:,2);
> booln = uint8(n~=128);
> m{1,j} = sum(booln .* n,1) ./ sum(booln,1);
> end
>
> In a nutshell I have a series of images, this opens each one, applies
> the colour transformation, and finds the means of the columns in
> (:,:,2), with a bit of code so the averaging ignores the whitespace.
>
> Trouble is this is taking in the order of 5-10 minutes depending on
> how many image files I have to loop through.
>
> What I'd prefer to do is open the individual images, concatenate them
> first, then carry out the colour transformation, which I think would
> be a lot quicker.
>
> I was attempting to do this but ran into the problem that the image
> files are various sizes, so I thought of padding them all out to the
> same size, but creating 'padding' of the right dimensions then
> applying it to each image prior to concatenating was making the
> program just as slow.
>
> Does anyone have any suggestions as to how I might go about improving
> this piece of code in any way shape or form?

a couple of questions:

Have you tried running the code through the profiler to figure out
where the time is being spent? You're probably correct that it's
being spent in applycform, but you may as well be certain before
expending any effort.

Also, by "concatenate" do you mean create a 3D matrix consisting of
"stacked" 2D images? how do you perform the padding?

PAUL MCGUIRE

2007-06-28, 7:17 pm

Paul McGuire wrote:
>
>
> Hi all, I've got this piece of code in a little program of mine:
>
> %loop filenames and do calculation
> for j=1:numfiles
> x=imread(strcat(direc,e{j},'.jpg'),'jpg');
> LAB = applycform(x,M);
> n =LAB(:,:,2);
> booln = uint8(n~=128);
> m{1,j} = sum(booln .* n,1) ./ sum(booln,1);
> end
>
> In a nutshell I have a series of images, this opens each one,
> applies
> the colour transformation, and finds the means of the columns in
> (:,:,2), with a bit of code so the averaging ignores the
> whitespace.
>
> Trouble is this is taking in the order of 5-10 minutes depending on
> how many image files I have to loop through.
>
> What I'd prefer to do is open the individual images, concatenate
> them
> first, then carry out the colour transformation, which I think
> would
> be a lot quicker.
>
> I was attempting to do this but ran into the problem that the image
> files are various sizes, so I thought of padding them all out to
> the
> same size, but creating 'padding' of the right dimensions then
> applying it to each image prior to concatenating was making the
> program just as slow.
>
> Does anyone have any suggestions as to how I might go about
> improving
> this piece of code in any way shape or form?

PAUL MCGUIRE

2007-06-28, 7:17 pm

snewdl wrote:
>
>
> On Jun 28, 4:21 am, "Paul McGuire" <mcgui...@hotmail.com>
wrote:
mine:[color=darkred]
> applies
in[color=darkred]
> whitespace.
depending[color=darkred]
> on
concatenate[color=darkred]
> them
> would
> image
to[color=darkred]
> the
> improving
>
> a couple of questions:
>
> Have you tried running the code through the profiler to figure out
> where the time is being spent? You're probably correct that it's
> being spent in applycform, but you may as well be certain before
> expending any effort.
>
> Also, by "concatenate" do you mean create a 3D matrix consisting of
> "stacked" 2D images? how do you perform the padding?
>
>

I've run profiler and the bottleneck is definitely at the applycform
step. I've also run the code before it and the code after it
separately and that just takes a few seconds, so I've little doubt
that it's this loop that's holding everything up.

I'll try and explain what I mean by concatenate. The images are small
sections of a long object whose colour varies along its length.
Currently the function opens the image of a single section, does the
colorspace transformation then takes the mean of (:,:2) (as the data
are 3D and I want the mean of the second 'layer') then puts the
vector created by 'mean' into a cell. It then opens the next image,
does the calculation and puts that in the next cell etc. etc. I then
concatenate the vectors created by 'mean' and that essentially gives
me a single vector which is 'degree of redness' v distance.

I think it would be faster if I could join the images prior to doing
the colorspace transformation. So if I had 3 images of dimensions
(for arguments sake):
image 1: 1000,2000,3
image 2: 2000,4000,3
image 3: 1500,2500,3

In order to concatenate them horizontally they need to have the same
number of rows, the idea I had was to pad the images with less rows
out with some rows of zeros. But I couldn't quite get it to work
properly and it was taking an even longer time.

I've got the idea of taking the mean of each 'layer' (i.e. mean R, G
and B) which should return a vector for R, G and B which should yield
a series of (1xNx3) matrices which I can then concatenate and apply
the colour transformation.

I'm just trying to work out how to do it now.

paul mcguire

2007-06-28, 7:17 pm

>
> I've got the idea of taking the mean of each 'layer' (i.e. mean R,
> G
> and B) which should return a vector for R, G and B which should
> yield
> a series of (1xNx3) matrices which I can then concatenate and apply
> the colour transformation.
>
> I'm just trying to work out how to do it now.

Just done it and taking the mean prior to converting to LAB buggers
everything up. So it's back to the drawing board.

Peter Boettcher

2007-06-29, 7:14 pm

"Paul McGuire" <mcguirpa@hotmail.com> writes:

> Hi all, I've got this piece of code in a little program of mine:
>
> %loop filenames and do calculation
> for j=1:numfiles
> x=imread(strcat(direc,e{j},'.jpg'),'jpg');
> LAB = applycform(x,M);
> n =LAB(:,:,2);
> booln = uint8(n~=128);
> m{1,j} = sum(booln .* n,1) ./ sum(booln,1);
> end
>
> In a nutshell I have a series of images, this opens each one, applies
> the colour transformation, and finds the means of the columns in
> (:,:,2), with a bit of code so the averaging ignores the whitespace.
>
> Trouble is this is taking in the order of 5-10 minutes depending on
> how many image files I have to loop through.
>
> What I'd prefer to do is open the individual images, concatenate them
> first, then carry out the colour transformation, which I think would
> be a lot quicker.
>
> I was attempting to do this but ran into the problem that the image
> files are various sizes, so I thought of padding them all out to the
> same size, but creating 'padding' of the right dimensions then
> applying it to each image prior to concatenating was making the
> program just as slow.

The only thing that could be saved by concatenating images first is
the function call overhead of applycform. This is very unlikely to
make a big difference give the computations inside applycform.
Especially given the overhead of reading from disk.

Anyway, you're probably better off discovering what applycform
actually does, and deciding if there are any corners you can cut. It
looks like you only use one plane of output. Maybe by only computing
that plane, you can save a bunch of work.

-Peter

Stephan Junek

2007-06-29, 7:14 pm

Peter Boettcher wrote:
>
>
> "Paul McGuire" <mcguirpa@hotmail.com> writes:
>
mine:[color=darkred]
> applies
in[color=darkred]
> whitespace.
depending[color=darkred]
> on
concatenate[color=darkred]
> them
> would
> image
to[color=darkred]
> the
>
> The only thing that could be saved by concatenating images first is
> the function call overhead of applycform. This is very unlikely to
> make a big difference give the computations inside applycform.
> Especially given the overhead of reading from disk.
>
> Anyway, you're probably better off discovering what applycform
> actually does, and deciding if there are any corners you can cut.
> It
> looks like you only use one plane of output. Maybe by only
> computing
> that plane, you can save a bunch of work.
>
> -Peter
>

Instead of concatenating the images, can't you preallocate a matrix
(with zeros) that should be big enough to fit all your data and then,
depending on the size of each image just "fill" the matrix with the
images? This should make a big difference to the image-by-image
addition.

Sponsored Links