Home > Archive > PERL CGI Beginners > January 2005 > embedding dynamic images in html output
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
embedding dynamic images in html output
|
|
| Chad Gard 2005-01-21, 3:55 pm |
| Hello, all.
I have a CGI that generates a page from data in a database. I want to
include several graphs of data in the page.
I can use GD::Graph to create the images. But, the data sets are
rather large/awkward to try to send off to another CGI that I could use
with the IMG tag, and I really don't want to write images to files on
disk. I'd like to be able to embed the image data directly in the
output of the one CGI.
I have found an offhand reference to embedding the image data in an
OBJECT tag (in Perl Graphics Programming, page 374 of the first
edition). But cannot find any information about how to actually go
about doing that.
Where can I learn how, using CGI.pm and GD::Graph, I can embed my image
data directly in the mass of HTML I'm spewing out?
TIA
--
Chad A Gard
chad@millerbrooks.com
Miller Brooks, Inc.
11712 N. Michigan Rd
Zionsville, IN 46077 USA
317.873.8100 (phone)
317.873.8110 (fax)
| |
| David Dorward 2005-01-21, 3:55 pm |
| On Fri, Jan 21, 2005 at 09:43:40AM -0500, Chad Gard wrote:
> I have a CGI that generates a page from data in a database. I want to
> include several graphs of data in the page.
>
> I can use GD::Graph to create the images. But, the data sets are
> rather large/awkward to try to send off to another CGI that I could use
> with the IMG tag, and I really don't want to write images to files on
> disk.
How about passing it the data it needs to make a suitable database
request?
> I'd like to be able to embed the image data directly in the
> output of the one CGI.
This isn't really practical.
> I have found an offhand reference to embedding the image data in an
> OBJECT tag (in Perl Graphics Programming, page 374 of the first
> edition). But cannot find any information about how to actually go
> about doing that.
This? http://www.faqs.org/rfcs/rfc2397.html
Browser support isn't very good, Internet Explorer doesn't support it
(for instance).
--
David Dorward http://dorward.me.uk
| |
| Bob Showalter 2005-01-21, 3:55 pm |
| Chad Gard wrote:
> and I really don't want to write images to files on disk.
I think you should reconsider. This is really your best bet. Web clients and
servers are really optimized for this kind of thing. You should write all
the images out to disk files accessible through URL's and then emit IMG tags
with those URL's. The server will keep the connection open so the client can
pull down the data.
You do have to manage cleaning up the leftover disk files after some period
of time.
| |
| Chad Gard 2005-01-21, 3:55 pm |
|
On Jan 21, 2005, at 9:56 AM, David Dorward wrote:
>
> How about passing it the data it needs to make a suitable database
> request?
>
That's what I'll do if I can't make it work the way I want. However,
I'd like to be able to avoid the overhead of additional queries, and
additional browser connections to the web server. I already get all
the data I need for other parts of the page, but there could be as many
25 small graphs in the page, with 9 being typical.
>
> This? http://www.faqs.org/rfcs/rfc2397.html
>
> Browser support isn't very good, Internet Explorer doesn't support it
> (for instance).
No, that wasn't want I was referring to, but may indeed work for me,
provided I can make my image data small enough. This is for an
intranet, and I have complete control over the browsers used. And the
only browsers used are Mozilla and occasionally Safari, so lack of IE
support isn't a big deal.
Thanks for the pointer - not what I was looking for, but probably a
solution!
--
Chad A Gard
chad@millerbrooks.com
Miller Brooks, Inc.
11712 N. Michigan Rd
Zionsville, IN 46077 USA
317.873.8100 (phone)
317.873.8110 (fax)
| |
| Sean Davis 2005-01-21, 3:55 pm |
|
On Jan 21, 2005, at 10:05 AM, Chad Gard wrote:
>
> On Jan 21, 2005, at 9:56 AM, David Dorward wrote:
>
> That's what I'll do if I can't make it work the way I want. However,
> I'd like to be able to avoid the overhead of additional queries, and
> additional browser connections to the web server. I already get all
> the data I need for other parts of the page, but there could be as
> many 25 small graphs in the page, with 9 being typical.
>
Do these need to all be generated on-the-fly? Or is there a way to
create them first (one time a day, for example) and serve them as
static images?
You could also have your main script generate the images all at once
(at the same time as you are generating the HTML), put the graphics in
temp files, and then put the appropriate URLs in the img tags. This
will eliminate the overhead of calling a helper script 9-25 times for a
single page. If you are on an intranet, serving 25 small images should
be almost instantaneous.
Sean
| |
| Chris Devers 2005-01-21, 3:55 pm |
| On Fri, 21 Jan 2005, Chad Gard wrote:
> That's what I'll do if I can't make it work the way I want. However,
> I'd like to be able to avoid the overhead of additional queries, and
> additional browser connections to the web server. I already get all
> the data I need for other parts of the page, but there could be as
> many 25 small graphs in the page, with 9 being typical.
I think you're out of luck.
What you want, in essence, is a way to generate an HTML document that
contains, inline, all the data to compose the page, So instead of
<img src="foo.png" alt="Foo" height="42" width="100">
you seem to want something more like this:
<img alt="Foo" height="42" width="100">
<!-- lots of binary data here -->
</img>
This is not, as far as I know, remotely possible.
You may be able to get something similar with XHTML, or maybe a
combination of XHTML and SVG, which is a vector graphics format that
encodes the data to produce the image in an XML format. But with regular
old HTML that can be expected to work on any browser in contemporary
usage (which will include some users of IE 4+, Netscape 4+, etc), this
is going to be a dead end.
Why are you so worried about the additional queries? They don't really
produce a lot of overhead. If your concen is that the iniial request
gets all the data needs to generate everything, and you don't want to
replicate all that effort, you can look into some caching tricks:
* have the initial request generate and cache the images
* have a request for an image check the cache before attempting to
generate anything: if it's there, return it, if not, generate and cache
it for future use
* have the cache be pruned automatically by some combination of age (no
old data or unused data) or size (set a ceiling for growth)
I really think that this is your best available approach. Serving
everything in one document isn't really a viable option. If you want to
get better performance on the server, some kind of caching system is
probably the way to go. If the approach above is too complex, you can
consider setting up a caching proxy server between your server that
generates the pages and the client -- both apache and Squid can do this,
among others.
--
Chris Devers
| |
| Chad Gard 2005-01-21, 3:55 pm |
|
On Jan 21, 2005, at 10:22 AM, Sean Davis wrote:
> Do these need to all be generated on-the-fly? Or is there a way to
> create them first (one time a day, for example) and serve them as
> static images?
They need to be generated on-the-fly, as the data changes continually.
>
> You could also have your main script generate the images all at once
> (at the same time as you are generating the HTML), put the graphics in
> temp files, and then put the appropriate URLs in the img tags. This
> will eliminate the overhead of calling a helper script 9-25 times for
> a single page. If you are on an intranet, serving 25 small images
> should be almost instantaneous.
Yeah, though then I have to worry about cleaning up the temp directory,
and I'd like to avoid unnecessary writes to the hard drive (to avoid
directory damage possibilities, energy use, fragmentation, etc). It
logically makes sense to just send these things out as they're
generated.
--
Chad A Gard
chad@millerbrooks.com
Miller Brooks, Inc.
11712 N. Michigan Rd
Zionsville, IN 46077 USA
317.873.8100 (phone)
317.873.8110 (fax)
| |
| Sean Davis 2005-01-21, 3:55 pm |
|
On Jan 21, 2005, at 11:03 AM, Chad Gard wrote:
>
> On Jan 21, 2005, at 10:22 AM, Sean Davis wrote:
>
>
> They need to be generated on-the-fly, as the data changes continually.
>
>
>
> Yeah, though then I have to worry about cleaning up the temp
> directory, and I'd like to avoid unnecessary writes to the hard drive
> (to avoid directory damage possibilities, energy use, fragmentation,
> etc). It logically makes sense to just send these things out as
> they're generated.
>
It does make some sense logically (but there are reasons it doesn't in
the general sense), but, unfortunately, no one does it that way, so
support is lacking for this kinda thing.
As for writes to the hard drive, it seems like a modern hard drive
should be able to handle this load--how many hits a day are you
expecting? And for cleanup, you can just set up a cron job that runs
at 2:00 AM or something to remove all the files from your temp
directory (assuming there are not thousands of files generated a day
that might fill the drive), particularly if you are on an intranet and
there is a low-use period. The only danger (besides filling the drive)
is that a user's query fails in the couple of seconds between
generating the HTML and pulling down the images because the files were
deleted in the meantime.
Doing this probably won't take too long. I (and others) would
encourage you to try it and see that it doesn't work before embarking
on more exotic solutions, but let us know what works for you....
Sean
| |
| Paul Archer 2005-01-21, 3:55 pm |
| 11:03am, Chad Gard wrote:
>
> On Jan 21, 2005, at 10:22 AM, Sean Davis wrote:
>
>
> Yeah, though then I have to worry about cleaning up the temp directory, and
> I'd like to avoid unnecessary writes to the hard drive (to avoid directory
> damage possibilities, energy use, fragmentation, etc). It logically makes
> sense to just send these things out as they're generated.
>
Just a thought here: why do you have a hard drive if you're not going to use
it? And "energy use"? You're not serious, are you? If the hard drive is
spinning, the extra energy used to move the heads is very, very little.
| |
| Wiggins d Anconia 2005-01-21, 3:55 pm |
| > 11:03am, Chad Gard wrote:
>
once (at[color=darkred]
temp[color=darkred]
single[color=darkred]
almost[color=darkred]
directory, and[color=darkred]
directory[color=darkred]
makes[color=darkred]
> Just a thought here: why do you have a hard drive if you're not going
to use
> it? And "energy use"? You're not serious, are you? If the hard drive is
> spinning, the extra energy used to move the heads is very, very little.
>
If we are talking about energy use then surely the amount of energy
sucked in by the CPU to generate any kind of complex graphed graphic
will be higher than to move the heads on the disk. Out of curiousity do
you not use a web log? So isn't there a disk write for every request
anyways? How about a log on the database queries, running the query for
every HTML page hit is going to cause another write, never mind the
potential for errors to be written. Most likely the database also has
tuning to limit requests, what if you hit that limit while generating
your results, blocking someone needing to do something for real, when
all you had to do was serve up a static graphic generated 2 months ago.
And with the 8MB *cache* on some hard drives today there is a good
chance your popular 30kb graphic doesn't even have to come from the
platter...
Seems odd to question caching technology given that cpus (and even the
cache has a cache these days), gpus, harddrives, web servers, browsers,
etc, etc, etc. all use them...
http://danconia.org
| |
| Shaun Fryer 2005-01-21, 8:55 pm |
| > >Yeah, though then I have to worry about cleaning up the temp directory,
> Just a thought here: why do you have a hard drive if you're not going to
> use it? And "energy use"? You're not serious, are you? If the hard drive is
> spinning, the extra energy used to move the heads is very, very little.
Johnny come lately here...
Please excuse me if I've missed something, but...
Philosophical issues aside, why can't you just have the image src be a link to
your cgi with query_string:some_param=1, causing it to print the image with
Content-Type: image/type. I have a gallery script running on my site, and this
is how it does image resizing dynamically. It's probably inefficient, but it
does save me some unneeded cruft on the hdd (which was the rationalle at the
time).
FYI: If you do that, you may need to append a .gif or whathaveyou to the uri
to make IE happy. Apparently IE likes to ignore Content-Type in favour of the
dot-whatever at the end of the uri. Then you're using Moz/Safarri, so it
shouldn't matter.
PS. In all reality though, the other commenters do have a valid point.
--
=====================
Shaun Fryer
=====================
http://sourcery.ca/
ph: 416-544-9461
=====================
| |
| Chad Gard 2005-01-24, 8:55 pm |
| Thanks for tips/pointers, all.
I was unable to make the graphs consistently small enough to use the
data url scheme that David pointed me toward. For now, I opted for
passing enough off to another script to make another database request.
It's causes several redundant SQL queries, and further clogs the pipe
between the web server and the database server, but it's not horrible.
At least it'll work for now.
This does actually give a beneficial side effect compared to what would
happen if I were embedding the data in object tags. The page can draw
before the graphs are created, so the user can start reading the
numerical data, and the graphs draw in as they are created. I use an
alt tag that says "creating graph" to let the user know why there's a
big rectangle there with an image tag, while the image's title
attribute is set to something more descriptive (which allows the tool
tip to be accurate, rather than always say "creating graph". Not great
for anyone visually impaired, but, well, it's an intranet and none of
the current users are.)
--
Chad A Gard
chad@millerbrooks.com
Miller Brooks, Inc.
11712 N. Michigan Rd
Zionsville, IN 46077 USA
317.873.8100 (phone)
317.873.8110 (fax)
| |
| Paul Archer 2005-01-24, 8:55 pm |
| 1:49pm, Chad Gard wrote:
> Thanks for tips/pointers, all.
>
> I was unable to make the graphs consistently small enough to use the data url
> scheme that David pointed me toward. For now, I opted for passing enough off
> to another script to make another database request. It's causes several
> redundant SQL queries, and further clogs the pipe between the web server and
> the database server, but it's not horrible. At least it'll work for now.
>
If your main script has the data, why not pass that data off to the script
that is drawing the graphs, rather than have it hit the database again?
Paul
---------------------------------------
-- In 1555, Nostradamus wrote: --
-- Come the millennium, month 12, --
-- In the home of greatest power, --
-- The village idiot will come forth --
-- To be acclaimed the leader. --
---------------------------------------
|
|
|
|
|