Toki pona corpus project needs your help!
Posted: Mon Feb 01, 2010 7:30 pm
I've been working on doing some corpus linguistics for toki pona. In the last hour, I've downloaded 220KB of long (as in paragraph or longer) texts of toki pona. I've substantial samples from Japanese, English, Russian, French, and German speakers.
Much of it is public domain or CC, so I'll repost a zip file shortly. The rest is unknown license, but I'll make it available to anyone who wants it for computational linguistics work, contact me directly and we'll arrange an exchange in a dark alley where copyright doesn't apply. If this was all on a generous license, we could republish it all as a comprehensive toki pona anthology on lulu.
As you might know, toki pona had a burning-of-the-libraries-of-Alexandria moment when Geocities went down. All of the following fan sites are gone.
geocities.com/stephentoddpope/tokiponabible.html
geocities.com/yohsweb/
geocities.com/yves_prudhomme/toki_pona/
geocities.com/stephentoddpope/tokiponahome
geocities.com/girlinside123/toki.html
Not only did we lose some irreplaceable works of early toki pona culture, but we've lost the "tenpo ale la mi pali e lipu ni. lipu ni li pini ala." animated gifs. tan ni kin la mi pana anpa e telo oko.
Corpus Research Resources and how you can help
Stand and be counted. Please post medium or long texts you have written to the net, preferably with at least one version without interlinear-gloss (ie. translations between each sentence)
Set your work free. If you do post somewhere on the web, please post with a generous license, public domain, CC or GPL documentation so that we can copy your works willy nilly without a guilty conscience. Otherwise we have to buy the linguistic corpus data from our pot dealer in dark alleys.
Post your work to the forum. When you post to this forum, it becomes covered by a CC license per your TOS with jan Sonja.
alasa tawa nimi pi toki pona. I usually can find most toki pona on the net by using toki pona key words, many of them do not exist in any language. There are significant gaps though in finding toki pona from Asian countries, probably because the search engines I know of aren't indexing hangul, chinese, etc.
http://www.suburbandestiny.com/?p=639 <-- Have I missed any toki pona pages on the net?
Search engine restricted to site listed above: http://www.google.com/cse/home?cx=01053 ... rmsq7cfp8g
Much of it is public domain or CC, so I'll repost a zip file shortly. The rest is unknown license, but I'll make it available to anyone who wants it for computational linguistics work, contact me directly and we'll arrange an exchange in a dark alley where copyright doesn't apply. If this was all on a generous license, we could republish it all as a comprehensive toki pona anthology on lulu.
As you might know, toki pona had a burning-of-the-libraries-of-Alexandria moment when Geocities went down. All of the following fan sites are gone.
geocities.com/stephentoddpope/tokiponabible.html
geocities.com/yohsweb/
geocities.com/yves_prudhomme/toki_pona/
geocities.com/stephentoddpope/tokiponahome
geocities.com/girlinside123/toki.html
Not only did we lose some irreplaceable works of early toki pona culture, but we've lost the "tenpo ale la mi pali e lipu ni. lipu ni li pini ala." animated gifs. tan ni kin la mi pana anpa e telo oko.
Corpus Research Resources and how you can help
Stand and be counted. Please post medium or long texts you have written to the net, preferably with at least one version without interlinear-gloss (ie. translations between each sentence)
Set your work free. If you do post somewhere on the web, please post with a generous license, public domain, CC or GPL documentation so that we can copy your works willy nilly without a guilty conscience. Otherwise we have to buy the linguistic corpus data from our pot dealer in dark alleys.
Post your work to the forum. When you post to this forum, it becomes covered by a CC license per your TOS with jan Sonja.
alasa tawa nimi pi toki pona. I usually can find most toki pona on the net by using toki pona key words, many of them do not exist in any language. There are significant gaps though in finding toki pona from Asian countries, probably because the search engines I know of aren't indexing hangul, chinese, etc.
http://www.suburbandestiny.com/?p=639 <-- Have I missed any toki pona pages on the net?
Search engine restricted to site listed above: http://www.google.com/cse/home?cx=01053 ... rmsq7cfp8g