=begin pod :kind("Language") :subkind("Language") :category("tutorial")

=TITLE Entering unicode characters

=SUBTITLE Input methods for unicode characters in terminals, the shell, and editors

Raku allows the use of unicode characters as variable names.  Many
operators are defined with unicode symbols (in particular the L<set/bag
operators|/language/setbagmix#Set%2FBag_operators>) as well as some quoting
constructs.  Hence it is good to know how to enter these symbols into editors,
the Raku shell and the command line, especially if the symbols aren't
available as actual characters on a keyboard.

General information about entering unicode under various operating systems
and environments can be found on the Wikipedia L<unicode input page|https://en.wikipedia.org/wiki/Unicode_input>.

=head1 XCompose (Linux)
X<|XCompose>

Xorg includes digraph support using a
L<I<Compose key>|https://en.wikipedia.org/wiki/Compose_key#GNU.2FLinux> .
The default of C<AltGr + Shift> can be remapped to something easier such as
C<Capslock>. In I<GNOME 2> and I<MATE> this can be setup under
C<Preferences → Keyboard → Layouts → Options → Position of Compose Key>.
So, for example, to input C<»+«> you could type
C<< CAPSLOCK > > + CAPSLOCK < < >>

I<XCompose> allows customizing the digraph sequences using a C<.XCompose>
file and
L<https://github.com/kragen/xcompose/blob/master/dotXCompose> is an extremely
complete one. In I<GNOME>, I<XCompose> was overridden and replaced with a
hardcoded list, but it is possible to restore I<XCompose> by setting
C<GTK_IM_MODULE=xim> in your environment. It might be necessary to install
a xim bridge as well, such as C<uim-xim>.

=head2 Getting compose working in all programs

You may have issues using the compose key in all programs. In that case you can
try C<ibus>.

=for code :lang<shell>
input_module=xim
export GTK_IM_MODULE=$input_module
export XMODIFIERS=@im=$input_module
export QT_IM_MODULE=$input_module

If you want this to be for all users you can put this in a file C</etc/profile.d/compose.sh>,
which is the easiest way, since you won't have to deal with how different GUI
environments set up their environment variables.

If you use KDE you can put this file in C<~/.config/plasma-workspace/env/compose.sh>
and that should work. Other desktop environments will be different. Look up
how to set environment variables in yours or use the system-wide option above.

=head2 ibus
X<|ibus>

If you have problems entering high codepoint symbols such as B<🐧> using the
C<xim> input module, you can instead use ibus. You will have to install the ibus
package for your distribution. Then you will have to set it to start on load of your
Desktop environment. The command that needs to be run is:

=for code :lang<shell>
ibus-daemon --xim --verbose --daemonize --replace

Setting C<--xim> should also allow programs not using ibus to still use the xim
input method and be backward compatible.

=head3 KDE

If you are using KDE, open the start menu and type in “Autostart” and click B<Autostart>
which should be the first result. In the settings window that opens, click
B<Add program>, type in C<ibus-daemon> and click OK. Then go into the Application
tab of the window that pops up. In the C<Command> field, enter in the full
ibus-daemon command as shown above, with the C<--desktop> option set to
C<--desktop=plasma>. Click OK. It should now launch automatically when you
log in again.

=head3 How to enter Unicode characters using a two-key combination

Using the XCompose input method it is possible to enter Unicode characters using simple two-key combinations.
Here's an example of a configuration which would enable entering all the Unicode characters used by Raku.
This example uses both Super keys as C<dead keys>. For example, to enter the C<π> symbol one has to press and
release the right C<Super> (or "Windows") key, then press and release the C<p> key.

=for code :lang<shell>
<Super_R> <less>               : "«" guillemotleft
<Super_R> <greater>            : "»" guillemotright
<Super_R> <minus>              : "⁻"   U207B
<Super_R> <0>                  : "⁰"   U2070
<Super_R> <1>                  : "¹"   U00B9
<Super_R> <2>                  : "²"   U00B2
<Super_R> <3>                  : "³"   U00B3
<Super_R> <4>                  : "⁴"   U2074
<Super_R> <5>                  : "⁵"   U2075
<Super_R> <6>                  : "⁶"   U2076
<Super_R> <7>                  : "⁷"   U2077
<Super_R> <8>                  : "⁸"   U2078
<Super_R> <9>                  : "⁹"   U2079
<Super_R> <asterisk>           : "×"   U00D7
<Super_R> <slash>              : "÷"   U00F7
<Super_R> <E>                  : "𝑒"   U1D452
<Super_R> <p>                  : "π"   U03C0
<Super_R> <t>                  : "τ"   U03C4
<Super_R> <grave>              : "‘"   U2018
<Super_R> <apostrophe>         : "’"   U2019
<Super_R> <comma>              : "‚"   U201A
<Super_R> <colon>              : "“"   U201C
<Super_R> <quotedbl>           : "”"   U201D
<Super_R> <L>                  : "„"   U201E
<Super_R> <period>             : "…"   U2026
<Super_R> <bracketleft>        : "≡"   U2261
<Super_R> <bracketright>       : "≢"   U2262
<Super_L> <8>                  : "∞"   U221E
<Super_L> <O>                  : "∅"   U2205
<Super_L> <e>                  : "∈"   U2208
<Super_L> <E>                  : "∉"   U2209
<Super_L> <3>                  : "∋"   U220B
<Super_L> <numbersign>         : "∌"   U220C
<Super_L> <minus>              : "−"   U2212
<Super_L> <slash>              : "∖"   U2216
<Super_L> <o>                  : "∘"   U2218
<Super_L> <U>                  : "∩"   U2229
<Super_L> <u>                  : "∪"   U222A
<Super_L> <asciitilde>         : "≅"   U2245
<Super_L> <equal>              : "≠"   U2260
<Super_L> <less>               : "≤"   U2264
<Super_L> <greater>            : "≥"   U2265
<Super_L> <c>                  : "⊂"   U2283
<Super_L> <C>                  : "⊃"   U2283
<Super_L> <v>                  : "⊄"   U2284
<Super_L> <V>                  : "⊅"   U2285
<Super_R> <d>                  : "⊆"   U2286
<Super_R> <D>                  : "⊇"   U2287
<Super_R> <f>                  : "⊈"   U2288
<Super_R> <F>                  : "⊉"   U2289
<Super_R> <U>                  : "⊍"   U228D
<Super_R> <u>                  : "⊎"   U228E
<Super_L> <t>                  : "⊖"   U2296
<Super_L> <L>                  : "｢"   UFF62
<Super_L> <l>                  : "｣"   UFF63

One can add these lines to their ~/.XCompose file. To activate the changes one needs to exit their X session
and login back again.
Note that since Ubuntu Gnome uses one C<Super> key for its own purposes, one might want to substitute one or
both the Super_* keys with, for example, Meta_R.

=head1 WinCompose (Windows)
X<|WinCompose>

L<WinCompose|https://github.com/samhocevar/wincompose> adds
L<compose key|https://en.wikipedia.org/wiki/Compose_key> functionality to Windows.
It can be installed either via the L<WinCompose releases|https://github.com/samhocevar/wincompose/releases>
page on GitHub, or with the L<Chocolatey package manager|https://chocolatey.org/packages/wincompose>.

Once the program is installed and running, right click the tray icon and
select C<Options → Composing → Behavior → Compose Key> to set your desired key.

WinCompose has multiple sources to choose from in C<Options → Composing → Sequences>.
It is recommended to enable C<XCompose> and disable C<Xorg>, as there are a handful
of operators which C<Xorg> does not provide sequences for, and C<Xorg> also has
sequences which conflict with operator sequences present in C<XCompose>.
Sequences can be viewed by right clicking the tray icon and selecting C<Show Sequences>.
If you wish to add your own sequences, you can do so by either adding/modifying
C<.XCompose> in C<%USERPROFILE%>, or editing user-defined sequences in the options menu.

=head1 Terminals, shells, and editors:

=head2 XTerm

Unicode support is enabled in XTerm primarily by setting its C<utf8> and
C<utf8Fonts> options to C<1>, along with its C<locale> option to C<UTF-8>, in
C<~/.Xdefaults>. Here is a sample configuration that supports displaying enough
of unicode to program in Raku:

    =begin code :lang<.Xdefaults>
    XTerm*faceName:           xft:Noto Mono:style=Regular
    XTerm*faceNameDoublesize: xft:Noto Emoji:style=Regular
    XTerm*faceSize:           10
    XTerm*locale:             UTF-8
    XTerm*titleModes:         16
    XTerm*utf8:               1
    XTerm*utf8Fonts:          1
    XTerm*utf8Title:          true
    =end code

=head2 URxvt

Similarly to XTerm, unicode support is enabled in URxvt primarily by setting
its C<locale> option to C<en_US.UTF-8> in C<~/.Xdefaults>. Here is a sample
configuration that supports displaying enough of unicode to program in Raku:

    =begin code :lang<.Xdefaults>
    URxvt*font:              xft:Noto Mono:pixelsize=14:style=Regular,\
                             xft:Noto Emoji:pixelsize=14:style=Regular
    URxvt*letterSpace:       -1
    URxvt*locale:            en_US.UTF-8
    URxvt*skipBuiltInGlyphs: true
    =end code

=head2 Unix shell

At the bash shell, one enters unicode characters by using entering
C<Ctrl-Shift-u>, then the unicode code point value followed by enter.  For
instance, to enter the character for the element-of operator (∈) use the
following key combination (whitespace has been added for clarity):

    =begin code :lang<shell>
    Ctrl-Shift-u 2208 Enter
    =end code

This also the method one would use to enter unicode characters into the
C<raku> REPL, if one has started the REPL inside a Unix shell.

=head2 Screen

L<GNU Screen|https://www.gnu.org/software/screen/> does sport a B<digraph>
command but with a rather limited digraph table. Thanks to B<bindkey> and
B<exec> an external program can be used to insert characters to the current
screen window.

    =begin code :lang<screen>
    bindkey ^K exec .! digraphs
    =end code

This will bind control-k to the shell command digraphs. You can use
L<digraphs|https://github.com/gfldex/digraphs> if you prefer a Raku friendly
digraph table over L<RFC 1345|https://tools.ietf.org/html/rfc1345> or change it
to your needs.

=head2 Vim
X<|Vim>

In L<Vim|https://www.vim.org/>, unicode characters are entered (in
insert-mode) by pressing first C<Ctrl-V> (also denoted C<^V>), then C<u> and
then the hexadecimal value of the unicode character to be entered.  For
example, the Greek letter λ (lambda) is entered via the key combination:

    =begin code :lang<vim>
    ^Vu03BB
    =end code

You can also use C<Ctrl-K>/C<^K> along with a digraph to type in some
characters.  So an alternative to the above using digraphs looks like this:

    =begin code :lang<vim>
    ^Kl*
    =end code

The list of digraphs Vim provides is documented
L<here|http://vimdoc.sourceforge.net/htmldoc/digraph.html>; you can add
your own with the C<:digraph> command.

Further information about entering special characters in Vim can be found on
the Vim Wikia page about L<entering special characters|http://vim.wikia.com/wiki/Entering_special_characters>.

=head3 vim-raku

The L<vim-raku|https://github.com/Raku/vim-raku> plugin for Vim can be
configured to optionally replace ASCII based ops with their Unicode based
equivalents. This will convert the ASCII based ops on the fly while typing
them.

=head2 Emacs
X<|Emacs>

In L<Emacs|https://www.gnu.org/software/emacs/>, unicode characters are
entered by first entering the chord C<C-x 8 RET> at which point the
text C<Unicode (name or hex):> appears in the minibuffer.  One then enters
the unicode code point hexadecimal number followed by the enter key.  The
unicode character will now appear in the document.  Thus, to enter the Greek
letter λ (lambda), one uses the following key combination:

    =begin code :lang<emacs>
    C-x 8 RET 3bb RET
    =end code

Further information about unicode and its entry into Emacs can be found on
the L<Unicode Encoding Emacs wiki page|https://www.emacswiki.org/emacs/UnicodeEncoding>.

You can also use L<RFC 1345|https://tools.ietf.org/html/rfc1345> character
mnemonics by typing:

    =begin code :lang<emacs>
    C-x RET C-\ rfc1345 RET
    =end code

Or C<C-u C-\ rfc1345 RET>.

To type special characters, type C<&> followed by a mnemonic.
Emacs will show the possible characters in the echo area.
For example, Greek letter λ (lambda) can be entered by typing:

    =begin code :lang<emacs>
    &l*
    =end code

You can use C<C-\> to toggle
L<input method|https://www.gnu.org/software/emacs/manual/html_node/emacs/Select-Input-Method.html>.

Another L<input method|https://www.emacswiki.org/emacs/InputMethods>
you can use to insert special characters is
L<TeX|https://www.emacswiki.org/emacs/TeXInputMethod>.
Select it by typing C<C-u C-\ TeX RET>. You can enter a special character
by using a prefix such as C<\>. For example, to enter λ, type:

    =begin code :lang<tex>
    \lambda
    =end code

To view characters and sequences provided by an input method,
run the C<describe-input-method> command:

    =begin code :lang<tex>
    C-h I TeX
    =end code

=head1 Some characters useful in Raku

=head2 L<Smart quotes|https://en.wikipedia.org/wiki/Quotation_mark#Curved_quotes_and_Unicode>

These characters are used in different languages as quotation marks. In Raku
they are used as L<quoting characters|/language/quoting>

Constructs such as these are now possible:

    say ｢What?!｣;
    say ”Whoa!“;
    say „This works too!”;
    say „There are just too many ways“;
    say “here: “no problem” at all!”; # You can nest them!

This is very useful in shell:

    =begin code :lang<shell>
    raku -e 'say ‘hello world’'
    =end code

since you can just copy and paste some piece of code and not worry about quotes.

=head2 L<Guillemets|https://en.wikipedia.org/wiki/Guillemet>

These characters are used in French and German as quotation marks. In Raku
they are used as L<interpolation word quotes|/language/quoting#Word_quoting_with_interpolation_and_quote_protection:_qqww>,
L<hyper operators|/language/operators#Hyper_operators> and as an angle bracket
alternative in POD6.

=begin table
    symbol   unicode code point   ascii equivalent
    ------   ------------------   ----------------
    «        U+00AB               <<
    »        U+00BB               >>
=end table

Thus constructs such as these are now possible:

    say (1, 2) »+« (3, 4);     # OUTPUT: «(4 6)␤» - element-wise add
    [1, 2, 3] »+=» 42;         # add 42 to each element of @array
    say «moo»;                 # OUTPUT: «moo␤»

    my $baa = "foo bar";
    say «$baa $baa ber».raku;  # OUTPUT: «("foo", "bar", "foo", "bar", "ber")␤»

=head2 Set/bag operators

The L<set/bag operators|/language/setbagmix#Set%2FBag_operators> all
have set-theory-related symbols, the unicode code points and their ascii
equivalents are listed below.  To compose such a character, it is merely
necessary to enter the character composition chord (e.g. C<Ctrl-V u> in Vim;
C<Ctrl-Shift-u> in Bash) then the unicode code point hexadecimal number.

=begin table
    operator   unicode code point   ascii equivalent
    --------   ------------------   ----------------
    ∈          U+2208               (elem)
    ∉          U+2209               !(elem)
    ∋          U+220B               (cont)
    ∌          U+220C               !(cont)
    ≡          U+2261               (==)
    ≢          U+2262               !(==)
    ⊆          U+2286               (<=)
    ⊈          U+2288               !(<=)
    ⊂          U+2282               (<)
    ⊄          U+2284               !(<)
    ⊇          U+2287               (>=)
    ⊉          U+2289               !(>=)
    ⊃          U+2283               (>)
    ⊅          U+2285               !(>)
    ∪          U+222A               (|)
    ∩          U+2229               (&)
    ∖          U+2216               (-)
    ⊖          U+2296               (^)
    ⊍          U+228D               (.)
    ⊎          U+228E               (+)
=end table

=head2 Mathematical symbols

Wikipedia contains a full list of L<mathematical operators and symbols in unicode|https://en.wikipedia.org/wiki/Mathematical_operators_and_symbols_in_Unicode>
as well as links to their mathematical meaning.

=head2 Greek characters

Greek characters may be used as variable names.  For a list of Greek and
Coptic characters and their unicode code points see the L<Greek in Unicode Wikipedia article|https://en.wikipedia.org/wiki/Greek_alphabet#Greek_in_Unicode>.

For example, to assign the value 3 to π, enter the following in Vim
(whitespace added to the compose sequences for clarity):

    =begin code :allow<B> :lang<vim>
    my $B<Ctrl-V u 03C0> = 3;  # same as: my $π = 3;
    say $B<Ctrl-V u 03C0>;     # 3    same as: say $π;
    =end code

=head2 Superscripts and subscripts

A limited set of
L<superscripts and subscripts|https://en.wikipedia.org/wiki/Superscripts_and_Subscripts>
can be created directly in unicode by using the C<U+207x>, C<U+208x> and
(less often) the C<U+209x> ranges.  However, to produce a value squared (to
the power of 2) or cubed (to the power of 3), one needs to use C<U+00B2> and
C<U+00B3> since these are defined in the
L<Latin1 supplement Unicode block|https://en.wikipedia.org/wiki/Latin-1_Supplement_(Unicode_block)>.

Thus, to write the L<Taylor series|https://en.wikipedia.org/wiki/Taylor_series>
expansion around zero of the function C<exp(x)> one would input into e.g.
vim the following:

    =begin code :allow<B> :lang<vim>
    exp(x) = 1 + x + xB<Ctrl-V u 00B2>/2! + xB<Ctrl-V u 00B3>/3!
    + ... + xB<Ctrl-V u 207F>/n!
    # which would appear as
    exp(x) = 1 + x + x²/2! + x³/3! + ... + xⁿ/n!
    =end code

Or to specify the elements in a list from C<1> up to C<k>:

    =begin code :allow<B> :lang<vim>
    AB<Ctrl-V u 2081>, AB<Ctrl-V u 2082>, ..., AB<Ctrl-V u 2096>
    # which would appear as
    A₁, A₂, ..., Aₖ
    =end code

=end pod

# vim: expandtab softtabstop=4 shiftwidth=4 ft=perl6
