Unicode format guide
How to remove zero-width characters safely
- Paste a non-sensitive sample and read the audit. It separates zero-width spaces, join controls, and BOM characters.
- Start with Cautious Cleanup. Use Remove All only when ZWNJ and ZWJ are unnecessary; use Leading BOM for a suspected initial U+FEFF.
- Compare the output with the original, especially multilingual words, emoji, URLs, and identifiers.
Why invisible characters appear in text
Some Unicode characters affect joining, wrapping, or file interpretation without drawing a visible mark. They may arrive through copied web content, messaging apps, rich-text editors, exported spreadsheets, or files saved with a byte order mark. Two strings can therefore look identical while failing an exact comparison, username check, search, or data import.
Visible result: productcode
Stored input: product[U+200B]code
The hidden U+200B can prevent an exact match even though no gap is visible.Characters checked by this tool
| Code point | Name | Why it needs care |
|---|---|---|
U+200B | Zero Width Space | Allows a wrapping opportunity without a visible space. |
U+200C | Zero Width Non-Joiner | Can intentionally control letter joining in several writing systems. |
U+200D | Zero Width Joiner | Participates in joined scripts and many emoji sequences. |
U+2060 | Word Joiner | Prevents a line break without adding visible width. |
U+FEFF | BOM / Zero Width No-Break Space | May identify byte order at a file start or appear unexpectedly inside text. |
When cautious cleanup is appropriate
- Cleaning identifiers copied from web pages or documents.
- Removing invisible breaks from email addresses, URLs, or product codes.
- Preparing plain Latin text for exact comparison.
- Removing a stray BOM after importing a UTF-8 file.
When not to remove everything
- ZWNJ and ZWJ can carry meaning in Arabic-derived and Indic scripts.
- ZWJ is used in emoji such as family and profession sequences.
- A word joiner may have been inserted intentionally to control wrapping.
- Always compare the output when multilingual text is involved.
How the three modes differ
Cautious Cleanup removes the characters most often found as unwanted residue—U+200B, U+2060, and U+FEFF—but retains the two joining controls. Remove All deletes all five listed code points and is intended for data fields where exact visible characters matter more than shaping. Leading BOM changes only a U+FEFF at position zero, leaving every other occurrence untouched.
How to recognize an invisible-character problem
Suspect hidden characters when identical-looking values have different lengths, a copied identifier fails an exact match, a URL breaks unexpectedly, or search misses visible text. An initial U+FEFF can also appear as an unexpected parser character. Use the audit rather than deleting every unusual Unicode character.
Which characters should be removed?
| Situation | Safer starting choice | Reason |
|---|---|---|
| Plain identifiers or Latin text | Cautious Cleanup | Removes common residue but keeps join controls. |
| Possible BOM at the first position | Leading BOM | Changes only an initial U+FEFF and minimizes unrelated edits. |
| Arabic-derived or Indic scripts | Review manually | ZWNJ and ZWJ may control joining. |
| Emoji sequences | Keep ZWJ | Removing U+200D can split one composed emoji into separate symbols. |
| Known machine identifier | Remove All, then verify | The expected value must be known. |
Real-world zero-width character examples
A BOM before structured data
Text may begin with U+FEFF. Some systems treat it as an encoding marker; others expose it as a character. Leading BOM removes only that starting occurrence.
A hidden character inside an identifier
AB[U+200B]123 looks like AB123, but exact comparison sees different strings. Cautious Cleanup removes U+200B for comparison with a trusted value.
ZWJ inside an emoji
Many emoji use U+200D to combine symbols. Remove All can separate them, so keep join controls unless the destination requires a character-by-character identifier.
Multilingual text using ZWNJ
U+200C can intentionally control joining. Invisible does not mean unwanted; review multilingual output before removing all join controls.
What remains unchanged
Regular spaces, non-breaking spaces, tabs, line endings, punctuation, and all other characters are preserved. The transformation runs in the browser, and the live audit counts categories rather than exposing the submitted text elsewhere. For visible but non-breaking spacing, use the non-breaking space remover. For a wider introduction, read the Unicode whitespace guide.
Frequently asked questions
Can zero-width characters break a URL or exact match?
Yes. An invisible code point can change the stored string even when the text looks unchanged. Remove it only after confirming what the expected value should be. For a pasted address, identify which URL component holds the character first; the guide to spaces in links applies the same check to visible spaces.
Will removing ZWJ break emoji?
It can. U+200D joins components in many emoji sequences. Cautious Cleanup keeps ZWJ; Remove All deletes it.
Is a BOM the same as a zero-width space?
No. U+FEFF is commonly a byte order mark at the start of encoded text. U+200B is the Zero Width Space used as an invisible breaking opportunity.
Does this tool remove non-breaking spaces?
No. NBSP characters have spacing width and belong to a different category. Use the Non-Breaking Space tool to inspect U+00A0, U+2007, and U+202F, or read what NBSP means before deciding whether to convert or keep them.
Does it detect every invisible Unicode character?
No. The audit targets U+200B, U+200C, U+200D, U+2060, and U+FEFF. It is a focused text transformer, not a complete Unicode security scanner.