Limitations

* You can't declare whitespace characters of any kind as sortable glyphs using the single-string ("short form") declaration. This is, obviously, because in that declaration format, whitespace is reserved as the delimiter for glyphs and families. So if you want to have space, tab, CR, and/or LF be sortable glyphs, you just have to declare that with the long form (LoL-reference) format. See the sections on these formats, below.

* When you have Sort::ArbBiLex generate a new bi-level sort function based on a sort-order declaration, both levels of comparison obviously have the same sort-order declaration -- so you can't have Sort::ArbBiLex make a function where at one level "ch" counts as one glyph, and at the other, it counts as two; nor where it counts as a glyph in one position in one level, and at another position in the other level.

* When you declare a glyph as consisting of several characters, you're saying that several letters should be considered as one unit. However, you can't go the other way: you can't say that a single letter should be considered as a combination of glyphs. But I've seen some descriptions of German sort order that say that a-umlaut (etc) should be treated as if it were a literal "ae" -- i.e. an "a" glyph followed by an "e" glyph. This can't be done simply with Sort::ArbBiLex.

* Note that ArbBiLex-generated sort routines always start sorting (at both levels) with glyphs at the start of the string, and continue to the end. But some descriptions (like p138 of the Unicode Standard Version 3.0) of French sort order say that the the first level of sorting goes as you'd expect, start to finish, but a later level, ties between different accents are broken starting from the end, and working backwards. This can't be done simply with Sort::ArbBiLex. (But it's my experience that the difference is not significant, in the case of French data.)

* If you're using a pre-Unicode version of Perl: you cannot declare more than 255 glyph-groups (i.e., glyphs that sort the same at the first level), and no glyph-group can contain more that 255 glyphs each. However, it's fine if the total number of glyphs in all glyph-groups sums to greater than 255 (as in the case of a declaration for 30 glyph-groups with 10 glyphs each).

* This library makes no provision for overriding the builtin "sort" function. It's probably a bad idea to try to do so, anyway.

* If all of the glyphs in a given sort order are one character long, the resulting sorter function will be rather fast. If any of them are longer than that, it is rather slower. (This is because one-character mode does its work with lots of "tr///"'s, whereas "multi"-character mode (i.e., if any glyphs are more than one character long) uses lots of "s///"'s and hashes. It's as fast as I can make it, but it's still necessarily much slower than single-character mode. So if you're sorting 10,000 dictionary headwords, and you change your sort order from one that uses all one-character glyphs, to one where there's even just one two-character glyph, and you notice that it now takes 15 seconds instead of 3 before, now you know why.

* Remember, if this module produces a function that almost does what you want, but doesn't exactly, because of the above limitations, then you can be have it output the source (via "source_maker") of the function, and try modifying that function on your own.

"use Sort::ArbBiLex;"

"use Sort::ArbBiLex ( 'name', DECLARATION, ... );"

This is probably the only way most users will need to interact with this module; they probably won't need to call "Sort::ArbBiLex::maker(DECLARATION)" (much less "Sort::ArbBiLex::source_maker(DECLARATION)"!) directly.

Unless your sort-order declarations are variables, you can simply use this "use Sort::ArbBiLex (...)" syntax. Feel free to skip ahead to the "Values for DECLARATION" section.

The function Sort::ArbBiLex::maker(DECLARATION)

  Sort::ArbBiLex::maker(DECLARATION)

This will make a sort function, based on the contents of DECLARATION. The return value is an anonymous subroutine reference. While you can store this just like any another anonymous subroutine reference, you probably want to associate it with name, like most functions. To associate it with the symbol "fulani_sort" in the current package, do this:

  *fulani_sort = Sort::ArbBiLex::maker($my_decl);

Then you can call "fulani_sort(@whatever)" just like any other kind of function, just as if you'd defined "fulani_sort" via:

  sub fulani_sort {
    ...blah blah blah...
  }

As you might expect, you can specify a package, like so:

  *MySorts::fulani_sort = Sort::ArbBiLex::maker($my_decl);

If you don't know what "*thing = EXPR" means or how it works, don't worry, just use it -- or duck the whole issue by using the ""use Sort::ArbBiLex ('fulani_sort', DECL);"".

Actually, there's a minor difference between the various ways of declaring the subroutine "fulani_sort": if you declare it via a call to this:

  *fulani_sort = Sort::ArbBiLex::maker($my_decl);

then that happens at runtime, not compile time. However, compile-time is when Perl wants to know what subs will exist if you want to be able to call them without parens. I.e., this:

  @stuff = fulani_sort @whatever;  # no parens!

will cause all sorts of angry error messages, which you can happily avoid by simply adding a "forward declaration" at some early point in the program, to express that you're goung to want to use "fulani_sort" as a sub name:

  sub fulani_sort;  # yup, just that!
  ...later...
  *fulani_sort = Sort::ArbBiLex::maker($my_decl);
  ...later...
  @stuff = fulani_sort @whatever;  # no parens!

And then all should be well.

The short story is to use the ""use Sort::ArbBiLex ('fulani_sort', ...)"" syntax whenever possible (at which point you're free to omit parens, since the "use" makes it happen at compile-time, not runtime).

But when you can't use the ""use Sort::ArbBiLex ('fulani_sort', ...)"" syntax, and you need to use a ""*foo = ..."" syntax instead (which is usually necessary if your declaration is a variable, instead of a literal), then either add a ""sub fulani_sort;"" line to your program; or just be sure to use parens on every call to the "fulani_sort" function.

See also: perlsub, for the whys and wherefors of function protoptying, if you want all the scary details.

The function Sort::ArbBiLex::source_maker(DECLARATION)

Short-form DECLARATION values

Formally, lines in the short-form declaration string are separated by carriage returns and/or linefeeds. Each line consists of glyphs separated by whitespace (other than CRs or LFs). Lines that are empty (i.e., which contain no glyphs) are ignored. A declaration that contains no glyphs at all is illegal, and causes a fatal error. A "glyph" is any sequence of non-whitespace characters. No glyph can appear more that once in the declaration, or it's a fatal error.

A degenerate case of there being only one glyph-family with many glyphs in it (i.e., a one-level sort instead of a bi-level), like this:

  Sort::ArbBiLex::maker('fulani_sort',
    "a A c C c' C' e E h H x X i I : l L n N r R s S u U z Z zh Zh ZH"
  );

is actually treated as if it were that many glyph-families with only one glyph in each. This in an internal optimization.

PLEASE NOTE that any characters that are in the data being sorted but which do not appear in the sort order declaration (neither as themselves, or as part of glyphs) are treated as if they are not there. In other words, given the sort order in the above example, if you had "David" as an item to sort, it would sort just as if it were "ai" -- since "D", "v", and "d" aren't in that declaration. So think twice before deciding that certain letters "are not part of the alphabet" of the language in question.

Note also that if, say, "ch" is in the sort order, but "h" isn't, then an "h" not after a "c" (like in "helicopter" or "pushed") will not be counted for sake of sorting.

Long-form DECLARATION values

  use Sort::ArbBiLex ( 'fulani_sort',
   [
     [ "a", "A" ],
     [ "c", "C" ],
     [ "ch", "Ch", "CH" ],
     [ "ch'", "Ch'", "CH'" ],
     [ "e", "E" ],
     [ "l", "L" ],
     [ "lh", "Lh", "LH" ],
     [ "n", "N" ],
     [ "r", "R" ],
     [ "s", "S" ],
     [ "u", "U" ],
     [ "z", "Z" ],
   ]
  );

The main practical reason I provide this declaration is that, as discussed in the Limitations section, the short form doesn't allow you to declare whitespace characters of any kind as sortable glyphs using the single-string ("short form") declaration, because in that declaration format, whitespace is reserved as the delimiter for glyphs and families. But you can do it in the long form. In the above example, you'd just add a line before the one for a/A, like this:

  use Sort::ArbBiLex ( 'fulani_sort',
    [
     [ " ", "\t", "\cm", "\cj" ],  # whitespace characters.
     [ "a", "A" ],
    ...etc...

That'd make whitespace the first glyph family. The effect of this would be to make sorting sensitive to whitespace, such that "for", "for sure", and "forest" would sort in that order. It's my impression that most modern English dictionaries sort without respect to whitespace (so that that list, sorted, would be "for", "forest", "for sure"), but I also realize that that's not the only way to do it. In fact, sensitivity to whitespace seems an inevitable part of conventional sort orders for some languages.

A thought: Presumably the only place you'd want to put the whitespace family is at the start of the declaration. It'd be really strange in the middle or the end, I think.

A word of caution: Note that, if you have whitespace as a glyph family, "for sure" (with just the one space) and "for sure" (with two spaces inbetween) do not sort the same. You may expect that the sorter would magically collapse whitespace, seeing all sequences of whitespace as equal. Au contraire! They're glyphs, just like any others, so sorting "for sure" and "for sure" (two spaces) is totally analogous to sorting "ika" and "ikka".