Petition koders.com [sic] to add Lisp to it's search category.

A couple of ws ago I came upon http://www.koders.com/, which seems
to be a new search engine focused on indexing Free Software/Open
Source code repositories on the web. They do not have Lisp of any kind
as a language search category, but typing in the word "Lisp" for any
language returns 40 pages (maximum that the search engine will find
for any keyword) of hits for Lisp implementations (even some of the
Unix interface bits for SBCL written in C show up, which actually
shouldn't because they are licensed under the Public Domain which is
not one of the licenses that koders.com purports to search for). Other
languages like Ruby and TCL, with substantially less Free
Software/Open Source code available than in the various Lisp dialects,
are listed (hell, even Fortran is too!). I gave them some feedback*,
listing the various Lisp dialects, amounts of code licensed under
various licenses for them (I estimated there's about 1 million LOC in
the Public Domain for Common Lisp alone, and I don't think I'm too far
off on that), etc., but still have not received any reply.

Now a good question to ask is why should anybody care about
koders.com? Well, it took me just a few minutes to find some very
interesting C code, and I believe a code search engine like koders.com
can be a very useful resource as well. Besides, listing Lisp besides
such magnificent triumphs of programming language design as Visual
Basic and TCL is bound to give the Lisp community some positive
exposure. For that reason, I propose that you try out koders.com, and
if you find the service useful, tell them you want Lisp code to be
indexed as well.

This also brings another idea to mind. I think a Lisp-specific,
community-run source code search engine, targetted by Lisp dialect,
would be a much more useful resource for the various Lisp communities
than koders.com or any similar generic service, because it would
produce more targetted search results and would be much more
responsive to user feedback. If you're interested in such a project
(and especially if you have written large-ish search engines in Lisp),
please post your opinions. I have a little bit of experience with
searching~, and would be willing to do most of the work on such a
project (provided anybody is interested), but I can't afford to host
anything as large as I'm proposing. I will have a large block of time
to devote to this during the holidays, after I'm done failing through
final exams (which reminds me, time to get back to studying!), and I'm
sure this is the case for a lot of other people too, so hopefully if
the interest is here we can make some progress by early next year.

Vladimir

PS - Sorry for cross-posting, but I think this concerns both Lispers
and Schemers (and a few more communities besides).

* - http://www.koders.com/info.aspx?c=feedback

~ - http://voodoohut.homeunix.net/iz-search
I feel a constant need to apologize for the above, because if it takes
you 10 seconds to find something, it's not because my code sucks, but
because that's how long it takes the disk to spin up from it's
perpetual power-saving sleep.

Vladimir Sedach <(string-downcase (concatenate 'string last-name (subseq first-name 0 1)))@
cpsc.ucalgary.ca> writes:
> This also brings another idea to mind. I think a Lisp-specific,
> community-run source code search engine, targetted by Lisp dialect,
> would be a much more useful resource for the various Lisp communities
> than koders.com or any similar generic service, because it would
> produce more targetted search results and would be much more
> responsive to user feedback.

You mean, like http://www.cliki.net/
or http://community.schemewiki.org/
or http://www.emacswiki.org/cgi-bin/wiki
(all have a search box and give targetted search results).

--
__Pascal Bourguignon__                     http://www.informatimago.com/
The world will now reboot; don't bother saving your artefacts.

Pascal Bourguignon <spam@mouse-potato.com> writes:

> Vladimir Sedach <(string-downcase (concatenate 'string last-name (subseq f
irst-name 0 1)))@cpsc.ucalgary.ca> writes: 
>
> You mean, like http://www.cliki.net/
> or http://community.schemewiki.org/
> or http://www.emacswiki.org/cgi-bin/wiki
> (all have a search box and give targetted search results).

Take a look at http://www.koders.com/. It doesn't search just
anything, but specifically indexes *source code* from Free
Software/Open Source projects. I think it's a lot more useful than
just posting code snippets to a wiki (not that anyone does that on
Cliki), primarily because you don't have to have anyone lift a finger
to contribute (hopefully) working, useful code. Another place where it
would work really well is data mining useful
functions/macros/subsystems from older, unsupported code (there's a
lot of good stuff on the CMU AI repository, but you have to be willing
to dig).

Vladimir

Vladimir Sedach <(string-downcase (concatenate 'string last-name (subseq fir
st-name 0 1)))@cpsc.ucalgary.ca> writes:

> Pascal Bourguignon <spam@mouse-potato.com> writes:
> 
>
> Take a look at http://www.koders.com/. It doesn't search just
> anything, but specifically indexes *source code* from Free
> Software/Open Source projects. I think it's a lot more useful than
> just posting code snippets to a wiki (not that anyone does that on
> Cliki), primarily because you don't have to have anyone lift a finger
> to contribute (hopefully) working, useful code. Another place where it
> would work really well is data mining useful
> functions/macros/subsystems from older, unsupported code (there's a
> lot of good stuff on the CMU AI repository, but you have to be willing
> to dig).

It's not useful to index source code.

Google does it already.
Google: "listing-stream" defun
http://www.google.com/search?q=%22l...stream%22+defun

How often do you get hits into a web page showing source code?  Right,
never (unless you design the question on purpose) because the
identifiers are not significant.

On the other hand, wikis are indexed by human operators with comments
that are worthy of indexing, so when you search cliki for "binary
tree" for example, you get interesting, "targeted" results.

On the other hand, when you google with relevant keywords, you can get
hits pointing to sources:
Google: lisp lalr parser
http://www.google.com/search?q=lisp+lalr+parser

--
__Pascal Bourguignon__                     http://www.informatimago.com/
The world will now reboot; don't bother saving your artefacts.

Pascal Bourguignon <spam@mouse-potato.com> writes:
> It's not useful to index source code.

Well, this isn't the type of interest I was talking about, but I guess
now I have a different motivation to work on the project - build it to
prove Pascal wrong. :)

> Google does it already.
> Google: "listing-stream" defun
>         http://www.google.com/search?q=%22l...stream%22+defun

But Google does the same thing as wiki search - it just indexes
whatever code snippets are posted on web pages.

> How often do you get hits into a web page showing source code?  Right,
> never (unless you design the question on purpose) because the
> identifiers are not significant.

That's how I (entirely accidentally) found out about Isis~.

Of course everyone knows identifiers are not significant - all good
Lisp programmers name their variables and functions with various
concatenations of "foo" and "bar", and never comment their code.

> On the other hand, wikis are indexed by human operators with comments
> that are worthy of indexing, so when you search cliki for "binary
> tree" for example, you get interesting, "targeted" results.

Nevertheless, my previous criticism of having to do work applies. You
also seem to think that I'm proposing this as a replacement for wikis
- I think it's a resource in it's own category with very little
overlap with wikis.

Vladimir

~ - http://www.medialabeurope.org/isis/

Vladimir Sedach <(string-downcase (concatenate 'string last-name (subseq fir
st-name 0 1)))@cpsc.ucalgary.ca> writes:

> Pascal Bourguignon <spam@mouse-potato.com> writes: 
>
> Well, this isn't the type of interest I was talking about, but I guess
> now I have a different motivation to work on the project - build it to
> prove Pascal wrong. :)
> 
>
> But Google does the same thing as wiki search - it just indexes
> whatever code snippets are posted on web pages.
> 
>
> That's how I (entirely accidentally) found out about Isis~.
>
> Of course everyone knows identifiers are not significant - all good
> Lisp programmers name their variables and functions with various
> concatenations of "foo" and "bar", and never comment their code.
> 
>
> Nevertheless, my previous criticism of having to do work applies. You
> also seem to think that I'm proposing this as a replacement for wikis
> - I think it's a resource in it's own category with very little
> overlap with wikis.

What I'm saying is that to find some interesting code, it's better to
search the comments about this code than the code itself.  Comment
about code are written in wikis, in code homepages or sites like
sourceforge, etc.

To take just sources (even with documentation if it exists) and try to
get a significantly searchable index, you'll have to resolve strong AI
first.

Actually, google works only by accident.  It was not obvious at all
before hand that it would give interesting results. Actually, some
users seem to have a hard time trying to get interesting results from
google: you have to choose your keywords with care and more than one.
You're betting that this accident could happen again with source code,
I bet not.

--
__Pascal Bourguignon__                     http://www.informatimago.com/
The world will now reboot; don't bother saving your artefacts.