Skip to content
The keyboard of an old Adler typewriter, capital letters on round black keys and a single red key at the top right.
"Adler Typewriter Keyboard Close-up" by Maksym Kozlenko, CC BY-SA 4.0, via Wikimedia Commons. Cropped to 3:2.

Anyone who keeps notes in Turkish hits this once: the word is in the note, and search does not find it. The cause is rarely a typo. The cause is four separate letters in the alphabet collapsing into two in code that was written with English in mind.

In Turkish, I and i are not the same letter

The Turkish alphabet has dotless I/ı and dotted İ/i as two distinct letters. ışık (light) and isik are different words; Irak and İrak are different places.

Most default lowercasing does not know this. In JavaScript, "I".toLowerCase() returns "i"; the same operation with the Turkish locale, "I".toLocaleLowerCase("tr"), returns "ı". A heading that reads ISTANBUL becomes either istanbul or ıstanbul depending on which path you took. Your user may type either.

Where lowercasing makes the string longer

This is the genuinely strange part. "İ".toLowerCase() does not return one letter: the result is i followed by a separate combining dot, a two-code-point string.

So "İSTANBUL".toLowerCase() is nine characters, not eight, and it does not compare equal to "istanbul". On screen the two strings look identical. The comparison returns false, the bug report says “search is broken”, and nobody thinks to check the length. With the Turkish locale, "İ".toLocaleLowerCase("tr") gives the single character you expected.

Where accent stripping stops half-way

The usual fix is to decompose the string and drop the combining marks. Çiğdem becomes cigdem, and someone without a Turkish keyboard can still find it.

The method works on five letters and fails on one. ç, ğ, ö, ş and ü decompose into a base letter plus a mark. Dotless ı is a single code point with no decomposition, so a mark-stripping normaliser leaves it alone: ışık does not become isik, it becomes ısık.

Run the same routine over İstanbul and you get Istanbul, with the capital dotless letter. Neither case produces an error. They produce a word that cannot be found.

Turkish collation is the stricter one

Here is the counterintuitive part: using the correct locale makes matching stricter, not more forgiving.

Compare with Intl.Collator at the lowest sensitivity and English treats c and ç as the same letter. Turkish does not, because in Turkish they genuinely are different letters, and the same goes for s against ş and o against ö. That same setting does treat i and İ as equal.

Sorting and searching are two jobs. Sorting wants alphabetical truth; searching has to settle for whatever the user could type.

The step that actually fixes it

Run both the content and the query through one search key. The decisive step in that key is the one that is alphabetically wrong: deliberately folding ı and i into the same letter.

Three steps. Lowercase with the Turkish locale, map ı to i, then decompose and strip the combining marks. Put IŞIK, ışık and isik through that chain and all three land on the same key, as do Çiğdem and cigdem.

A normaliser that skips the second step sends two spellings of one word to two different keys. The popular recipe of default lowercasing plus accent stripping does exactly that: IŞIK comes out as isik while ışık comes out as ısık. Type the word in capitals and you find it; type it in lower case and you don’t.

Don’t reuse this chain for sorting. There the right answer is Turkish alphabetical order, and a list that files ç under c looks wrong to a Turkish reader from the first glance. That split is what sits behind Turkish search not tripping over Turkish characters in Context: two representations of the same text, one forgiving and one exact.

All posts

GETTING STARTED

Open it in your browser now.

The web version is live and it is the whole thing — not a cut-down trial copy. If it isn't for you, close the tab and nothing is left behind.

  • No account needed
  • No card asked for
  • 14-day trial on your first purchase