APL - APL and OLAP

Author

APL and OLAP

Lee Dickey

2005-01-20, 3:56 pm

There is an interesting article that has recently appeared
about OLAP, which says that OLAP its origins to APL.

The author, Konrad Dramowicz, describes the idea
of "data cube". He says that there are five basic operations:

Slice
Dice
Roll-up
Drill down
Pivoting

which, as far as I can tell refer to various ways to select data
from a three dimensional data array.
I suspect that most APLers are familiar with these
concepts, but perhaps they do not use this vocabulary.

I found the article here:

http://www.directionsmag.com/articl...8b31a5
8e

If your news reader dies on the above line, I have broken it here,
so you might be able to piece it together.

http://www.directionsmag.com/article.php?
article_id=733&trv=1& PHPSESSID=023ab7197e3c93bdaa6d63a68b31a5
8e

Lee

Dick Bowman

2005-01-21, 8:55 am

ljdickey@pythagoras.math.uwaterloo.ca (Lee Dickey) wrote in
news:csoifd$32t$1@rumours.uwaterloo.ca:

> There is an interesting article that has recently appeared
> about OLAP, which says that OLAP its origins to APL.
>
> [... deleted ...]

This is true to a degree (APL and OLAP share multi-dimensionality, and
it's easy to make a start building your own OLAP with APL - or J - as the
basis).

When I looked at some "popular OLAP packages" some years ago what was
striking (to me, coming from an APL background) was how they struggled
dealing with multidimensional array concepts. Things were dressed down
to virtually disguise underlying mathematics - but you could see them
struggling to get where they needed to be. My sense at the time was that
they needed a few years and then they'd invent their own version of APL
(I don't know if they've done this yet).

I suspect that one reason they feel so clunky is that while
conceptualising and visualising multiple dimensions comes easily to APL
native-speakers, it does not come easily to the population at large, and
the vendors need to get this stuff used by people who are not pre-
selected for aptitude.

Something else we noticed at the time was that performance-wise the big
OLAP packages could not hold a candle to a custom APL solution for tasks
which were crucial to the specific application.

But, to be fair, the packages do seem to cater for one aspect of OLAP
data as native APL does not - the arrays are sparse. I haven't used J's
sparse arrays (and can't remember whether any of the APLs handle
sparseness explicitly) - but OLAP is really about sparse
multidimensionality. We have the algebraic notation, OLAP packages have
the sparsity.

I think the popularity and pricing of OLAP packages tells us a lot about
the marketing (and marketability) of APL.

Björn Helgason

2005-01-21, 8:55 am

"I think the popularity and pricing of OLAP packages tells us a lot
about
the marketing (and marketability) of APL."
What are the prices and what does it tell us?

2005-01-21, 8:55 pm

some years ago i was lucky enough to be on the receiving end of a
speed-lecture on OLAP delivered by dennis shasha of nyu.

based on what i learned that afternoon, i put together a small demo
in k, which is here:

http://www.nsl.com/papers/drilldown.htm

there are only a couple of basic constructs: a "fact" table, and
a set of "dimension" tables, which contain the axis hierarchies.
(in a production version, you would normalize these structures.)

a simple GUI lets you "drill down" (i.e. disaggregate) a cell (two-
dimensional) or a row or column (one-dimensional). in a fully-
featured GUI, you would probably allow for more than one logical
axis on each of the two visual axes.

axes can be sliced and pivoted, and operations are recorded in
the form of a dynamic SQL-like query statement. an interpreter
for such statements is easy to write.

the idea of rolling aggregation of transactions up the paths of
a tree is in some loose sense complementary to OLAP:

http://www.nsl.com/papers/rollup.htm

a very interesting, and very strange book on the subject is:

http://www.amazon.com/exec/obidos/t...=glance&s=books

i say "strange" because who would expect to find side-by-side
chapters on portfolio analysis and wittgenstein's tractatus?

"Dick Bowman" <bowman@apl.demon.co.uk> wrote in message news:Xns95E554F232435bowmanapldemoncouk@
158.152.254.254...
> ljdickey@pythagoras.math.uwaterloo.ca (Lee Dickey) wrote in
> news:csoifd$32t$1@rumours.uwaterloo.ca:
>
>
> This is true to a degree (APL and OLAP share multi-dimensionality, and
> it's easy to make a start building your own OLAP with APL - or J - as the
> basis).
>
> When I looked at some "popular OLAP packages" some years ago what was
> striking (to me, coming from an APL background) was how they struggled
> dealing with multidimensional array concepts. Things were dressed down
> to virtually disguise underlying mathematics - but you could see them
> struggling to get where they needed to be. My sense at the time was that
> they needed a few years and then they'd invent their own version of APL
> (I don't know if they've done this yet).
>
> I suspect that one reason they feel so clunky is that while
> conceptualising and visualising multiple dimensions comes easily to APL
> native-speakers, it does not come easily to the population at large, and
> the vendors need to get this stuff used by people who are not pre-
> selected for aptitude.
>
> Something else we noticed at the time was that performance-wise the big
> OLAP packages could not hold a candle to a custom APL solution for tasks
> which were crucial to the specific application.
>
> But, to be fair, the packages do seem to cater for one aspect of OLAP
> data as native APL does not - the arrays are sparse. I haven't used J's
> sparse arrays (and can't remember whether any of the APLs handle
> sparseness explicitly) - but OLAP is really about sparse
> multidimensionality. We have the algebraic notation, OLAP packages have
> the sparsity.
>
> I think the popularity and pricing of OLAP packages tells us a lot about
> the marketing (and marketability) of APL.
>

stevemansour@yahoo.com

2005-01-24, 3:57 pm

I presented a paper "Houses Windows and DOHRs" at APL 2000 in Berlin.
The paper defines Descriptive Objects of High Rank (DOHR's) which are
somewhat analagous to data cubes. DOHR's have Selection, Summary and
Orientation operations which are analogous to the Slice, Roll-up and
Pivoting operations described in the OLAP paper.

houb

2005-01-24, 3:57 pm

This is of course not the only article that says the origin of olap lays in
apl: for several years now there is a similar statement in the olap report :
http://www.olapreport.com/origins.htm

you may also be interested in my olap-like apl application
http://home.wanadoo.nl/houb/IDA.htm
(work in - slow - progress)

/Bernard

"Lee Dickey" <ljdickey@pythagoras.math.uwaterloo.ca> schreef in bericht
news:csoifd$32t$1@rumours.uwaterloo.ca...
> There is an interesting article that has recently appeared
> about OLAP, which says that OLAP its origins to APL.
>
> The author, Konrad Dramowicz, describes the idea
> of "data cube". He says that there are five basic operations:
>
> Slice
> Dice
> Roll-up
> Drill down
> Pivoting
>
> which, as far as I can tell refer to various ways to select data
> from a three dimensional data array.
> I suspect that most APLers are familiar with these
> concepts, but perhaps they do not use this vocabulary.
>
> I found the article here:
>
> http://www.directionsmag.com/articl...8b31a5
8e
>
> If your news reader dies on the above line, I have broken it here,
> so you might be able to piece it together.
>
> http://www.directionsmag.com/article.php?
> article_id=733&trv=1& PHPSESSID=023ab7197e3c93bdaa6d63a68b31a5
8e
>
> Lee
>

Sponsored Links