When to use a tool and when to write code
Use a browser tool when the job is bounded: a block of text pasted once, a value checked before it goes into a form, a document cleaned before publishing. There is nothing to install, the live audit shows which hidden characters were present, and the transformation is deterministic. Use code when the job repeats: a folder of files, a column that arrives daily, a build step, or any process that another person or a server needs to run without you.
The two approaches also fail differently. A tool keeps you in the loop, so you notice when a cleanup joins two words. A script applies the same rule to a thousand rows whether or not that rule was right, so it needs test cases. Many people use both: the tool to work out which characters are involved, then a script that targets exactly those.
Python
Python 3 strings are Unicode, and the re module treats \s as Unicode whitespace by default, so patterns match NBSP and most typographic spaces. The string methods behave as follows:
| Expression | Effect | Watch out for |
|---|---|---|
s.strip() | Removes leading and trailing whitespace, including Unicode spaces. | Only the outer edges of the whole string; interior lines untouched. |
" ".join(s.split()) | Collapses every run of whitespace to one space and trims ends. | Also collapses newlines, so line structure is lost. |
re.sub(r"\s+", " ", s) | Collapses whitespace runs to one space. | \s includes line separators. The pattern r"[^\S\n]+" keeps only LF; it still matches CR, VT, FF, NEL, LS and PS. Use an explicit spacing class if those controls must stay. |
s.replace("\u00A0", " ") | Converts only NBSP to a regular space. | Misses U+202F and U+2007; chain replacements or use a regex class. |
"".join(s.split()) | Removes all whitespace entirely. | Joins words and values with no separator. |
s.encode().decode("utf-8-sig") | Strips a leading BOM (U+FEFF). | Only the BOM at the start; not zero-width characters mid-text. |
For a Python cleanup that rebuilds line structure, use "\n".join(line.strip() for line in s.splitlines()). This is not an exact copy of the browser tool: str.splitlines() also splits on U+2028, U+2029 and other separators, and joining writes LF while losing an ending trailing break. The browser tool preserves those controls and trims supported spacing only at its stored LF row edges.
JavaScript
JavaScript follows a similar model. String.prototype.trim() removes whitespace and line terminators from both ends, and the regex \s class matches U+0020, tab, the Unicode space separators including U+00A0, and line terminators.
s.replace(/\s+/g, " ").trim()collapses runs to one space across the whole string, newlines included.- This collapses the homepage's supported spacing without changing CR, LF, VT, FF, NEL, LS or PS. Add per-line edge trimming if needed. The shorter
s.replace(/[\t \u00A0\u1680\u2000-\u200A\u202F\u205F\u3000\uFEFF]+/gu, " ")/[^\S\r\n]+/gpattern keeps CR/LF but also matches VT, FF, LS and PS, so it is not equivalent. - This removes the zero-width space, word joiner, and BOM. Add
s.replace(/[\u200B\u2060\uFEFF]/g, "")\u200C\u200Dto the class only when the text has no joined scripts or emoji. s.normalize("NFKC")folds many compatibility spaces, including U+00A0 and U+2007, to a regular space as a side effect. Use it deliberately, because it also changes other characters.
Command line
Shell tools operate on bytes unless the locale is set, so multi-byte Unicode spaces are the main hazard.
sed 's/[[:space:]]\{1,\}/ /g'collapses ASCII whitespace runs; it will not see U+00A0 as space in the C locale.tr -s '[:space:]' ' 'squeezes runs but also flattens newlines.sed 's/[ \t]*$//'trims trailing spaces and tabs per line, a common pre-commit cleanup.- For Unicode spaces, use
perl -CSD -pe 's/\s+/ /g'or a Python one-liner, which decode UTF-8 first.
Spreadsheets
Excel and Google Sheets TRIM is the most common source of confusion. It removes leading, trailing, and repeated U+0020 spaces only. It does not touch a non-breaking space (character 160), which is exactly the character that arrives when data is pasted from a web page. The fix is to substitute it first:
=TRIM(SUBSTITUTE(A1, CHAR(160), " "))
Also remove a zero-width space:
=TRIM(SUBSTITUTE(SUBSTITUTE(A1, CHAR(160), " "), UNICHAR(8203), ""))For a whole sheet this is tedious, which is a case where pasting the affected column into the NBSP tool or the space remover and pasting it back is faster than building nested formulas.
Side-by-side comparison
| Need | Browser tool | Python / JS | sed / tr | Spreadsheet |
|---|---|---|---|---|
| Collapse multiple spaces | Yes, one step | Yes | Yes (ASCII) | TRIM |
| Remove all spaces | Yes | Yes | Yes | SUBSTITUTE |
| Handle NBSP and Unicode spaces | Yes, detected and counted | Yes with the right pattern | Only with perl or a UTF-8 locale | Only via SUBSTITUTE(CHAR(160)) |
| Remove zero-width characters | Yes, with a cautious mode | Yes with an explicit class | Hard | SUBSTITUTE(UNICHAR(8203)) |
| Per-line trim, keep line breaks | Yes | Yes with a loop | Yes with an anchor | Per cell only |
| Repeatable on many files | No | Yes | Yes | Limited |
| No environment or install | Yes | No | Shell needed | App needed |
| Shows what was hidden | Yes, live audit | Only if you print it | No | No |
A short decision guide
- One block of text, or a value you need to inspect: use a tool here and read the audit.
- A column in a spreadsheet: substitute CHAR(160) then TRIM, or round-trip the column through a tool.
- Many files, a data pipeline, or a build step: write Python or JavaScript with an explicit whitespace class and test cases.
- A quick trailing-space cleanup in version control: a
sedor editor "trim trailing whitespace" setting. - Unsure which characters are involved: diagnose with a tool first, then encode that finding in code.
The definitions used here follow the WHATWG strip and collapse ASCII whitespace algorithm and Unicode Annex #14. For the full character list, see the whitespace character reference; for data-specific cases, see whitespace in data cleaning.