<!-- GENERATED by docs/sphinx/bin/20-reference (p5) from Unicode/Normalize/mod.rs. Do not edit. -->

```{index} single: Unicode::Normalize
```

# Unicode::Normalize

```{pperl-module-badges} Unicode::Normalize
```

`Unicode::Normalize` - the Unicode normalization forms NFD, NFC, NFKD
and NFKC (UAX #15), and the FCD and FCC forms of UTN #5.

```perl
use Unicode::Normalize qw(NFD NFC NFKC normalize checkNFC getCombinClass getCanon);

binmode STDOUT, ':encoding(UTF-8)';
my $s = "Cafe\x{301} \x{FB01}";              # e + combining acute, "fi" ligature
printf "NFC  %s (%d chars)\n", NFC($s), length NFC($s);   # Café ﬁ (6 chars)
printf "NFD  %d chars\n", length NFD("Caf\x{e9}");        # 5 chars
printf "NFKC %s\n", NFKC($s);                             # Café fi
print  "same: ", (normalize('C', $s) eq NFC($s) ? "yes" : "no"), "\n";
print  "checkNFC: ", (defined checkNFC($s) ? checkNFC($s) || 'NO' : 'MAYBE'), "\n";
printf "class of U+0301: %d, U+00E9 = %vX\n", getCombinClass(0x301), getCanon(0xE9);
# class of U+0301: 230, U+00E9 = 65.301
```

Two strings that look the same can be different sequences of
characters: `é` is one character, U+00E9, or two, `e` and the
combining acute U+0301. Normalizing both to the same form makes them
compare equal. NFD and NFKD decompose every character as far as it
goes; NFC and NFKC decompose and then compose again, giving the short
form. The K forms also replace compatibility characters - ligatures,
full-width forms, superscripts - by their plain equivalents, which
loses formatting; NFC is the form to store and compare text in.

The data are those of the Unicode version perl itself was built with
(17.0.0 for perl 5.44), so normalization agrees with perl's own
character properties.

This is a native implementation of `Unicode::Normalize` 1.32; its
`Unicode/Normalize.pm` is used unchanged. `use Unicode::Normalize`
exports `NFC`, `NFD`, `NFKC` and `NFKD`; every other function named on
this page is exported on request. The `.pm` adds, over these
functions:

- `normalize($form, $string)`, where `$form` is `C`, `D`, `KC`, `KD`
  (or `NFC` ...), `FCD` or `FCC`, and `check($form, $string)` for
  the quick checks.
- `FCD($string)`: the string unchanged when it is already in FCD,
  else a string in FCD.
- `normalize_partial($form, $unprocessed)` and `NFD_partial`,
  `NFC_partial`, `NFKD_partial`, `NFKC_partial`: normalize what can
  be normalized now and leave the rest in `$unprocessed`, for text
  that arrives in pieces (see `splitOnLastStarter`).

A code point argument is converted as an unsigned integer: pass a
plain non-negative integer.

## Differences from upstream

None.

## Functions

### Normalization forms

#### `NFD`

`NFD($string)`: Normalization Form D, the canonical decomposition with the combining marks in canonical order. `NFKD($string)` is Form KD, the same with compatibility decomposition. Both exported by default.

Also available as `Unicode::Normalize::NFKD`.

#### `FCC`

`NFC($string)`: Normalization Form C, canonical decomposition followed by canonical composition - the composed form, and the usual one for storing and comparing text. `NFKC($string)` is Form KC, from the compatibility decomposition; `FCC($string)` is UTN #5's "Fast C Contiguous" form. `NFC` and `NFKC` are exported by default.

Also available as `Unicode::Normalize::NFC`, `Unicode::Normalize::NFKC`.

### Quick checks

#### `checkNFD`

`checkNFD($string)`, `checkNFKD($string)`: the quick check of UAX #15, 1 when the string is in NFD (NFKD) and `""` when it is not. For these forms the answer is always definite.

Also available as `Unicode::Normalize::checkNFKD`.

#### `checkNFC`

`checkNFC($string)`, `checkNFKC($string)`: the quick check - 1 when the string is in NFC (NFKC), `""` when it is not, and `undef` for "maybe": a combining mark the check cannot decide on alone. `checkNFC("A\x{301}")` and `checkNFC("B\x{301}")` both say maybe, though only the first is not in NFC; for a definite answer compare `$string eq NFC($string)`.

Also available as `Unicode::Normalize::checkNFKC`.

#### `checkFCC`

`checkFCD($string)`: 1 when the string is in FCD, `""` when not. `checkFCC($string)`: the same for FCC, with `undef` for "maybe"; a string not in FCD is never in FCC.

Also available as `Unicode::Normalize::checkFCD`.

### Building blocks

#### `decompose`

`decompose($string [, $compat])`: each character replaced by its canonical decomposition, or by its compatibility decomposition when `$compat` is true. The combining marks are not put in canonical order: `reorder(decompose($s))` is `NFD($s)`, and `reorder(decompose($s, 1))` is `NFKD($s)`.

#### `reorder`

`reorder($string)`: every run of combining marks put in canonical order. Joined NFD (or NFKD) strings are NFD again after `reorder`: `reorder(join '', @nfd_strings)`.

#### `compose`

`compose($string)`: canonical composition, without decomposing first; `compose($nfd)` is the NFC of an NFD string, `compose($nfkd)` the NFKC of an NFKD one. `composeContiguous($string)` composes only characters that are next to each other, which is how FCC is made.

Also available as `Unicode::Normalize::composeContiguous`.

#### [`splitOnLastStarter`](Normalize/splitOnLastStarter)

`splitOnLastStarter($normalized)`: two strings, the part before the last starter (a character of combining class 0) and the rest. The rest may still change when more text follows, so text normalized in pieces is put together as:

### Character data

#### `getCombinClass`

`getCombinClass($code_point)`: the canonical combining class, 0 for a starter (230 for U+0301 COMBINING ACUTE ACCENT).

#### `isExclusion`

`isExclusion($code_point)`: whether it is a composition exclusion, a character composition never produces.

#### `isSingleton`

`isSingleton($code_point)`: whether it decomposes canonically to a single other character.

#### `isNonStDecomp`

`isNonStDecomp($code_point)`: whether its decomposition starts with a non-starter (a character of combining class other than 0).

#### `isComp2nd`

`isComp2nd($code_point)`: whether it can compose with the character before it (Hangul included, exclusions not). `isNFC_MAYBE` and `isNFKC_MAYBE` are the same property under its quick-check names, `NFC_Quick_Check=Maybe` and `NFKC_Quick_Check=Maybe`.

Also available as `Unicode::Normalize::isNFC_MAYBE`, `Unicode::Normalize::isNFKC_MAYBE`.

#### `isNFD_NO`

`isNFD_NO($code_point)`: the derived property `NFD_Quick_Check=No` - whether it decomposes canonically. `isNFKD_NO` is `NFKD_Quick_Check=No`: whether it decomposes at all.

Also available as `Unicode::Normalize::isNFKD_NO`.

#### `isComp_Ex`

`isComp_Ex($code_point)`: the derived property Full_Composition_Exclusion - an exclusion, a singleton or a non-starter decomposition. `isNFC_NO` is the same property, as `NFC_Quick_Check=No`; `isNFKC_NO` is `NFKC_Quick_Check=No`, which adds every character whose compatibility decomposition differs from its canonical one (U+FB01, the "fi" ligature).

Also available as `Unicode::Normalize::isNFC_NO`, `Unicode::Normalize::isNFKC_NO`.

#### `getComposite`

`getComposite($code_point, $next_code_point)`: the code point the two compose to (Hangul included, exclusions excluded), or `undef`.

#### `getCanon`

`getCanon($code_point)`: the full canonical decomposition as a string, or `undef` for a character that does not decompose canonically. `getCompat($code_point)` is the same for the compatibility decomposition. Hangul syllables decompose to their jamo.

Also available as `Unicode::Normalize::getCompat`.

```{toctree}
:hidden:
:maxdepth: 1

Normalize/splitOnLastStarter
```