When to use a tool and when to write code
Use a browser tool when the job is bounded: a block of text pasted once, a value checked before it goes into a form, a document cleaned before publishing. There is nothing to install, the live audit shows which hidden characters were present, and the transformation is deterministic. Use code when the job repeats: a folder of files, a column that arrives daily, a build step, or any process that another person or a server needs to run without you.
The two approaches also fail differently. A tool keeps you in the loop, so you notice when a cleanup joins two words. A script applies the same rule to a thousand rows whether or not that rule was right, so it needs test cases. Many people use both: the tool to work out which characters are involved, then a script that targets exactly those.
Python
Python 3 strings are Unicode, and the re module treats \s as Unicode whitespace by default, so patterns match NBSP and most typographic spaces. The string methods behave as follows:
| Expression | Effect | Watch out for |
|---|---|---|
s.strip() | Removes leading and trailing whitespace, including Unicode spaces. | Only the outer edges of the whole string; interior lines untouched. |
" ".join(s.split()) | Collapses every run of whitespace to one space and trims ends. | Also collapses newlines, so line structure is lost. |
re.sub(r"\s+", " ", s) | Collapses whitespace runs to one space. | \s includes \n; use r"[^\S\n]+" to keep newlines. |
s.replace("\u00A0", " ") | Converts only NBSP to a regular space. | Misses U+202F and U+2007; chain replacements or use a regex class. |
"".join(s.split()) | Removes all whitespace entirely. | Joins words and values with no separator. |
s.encode().decode("utf-8-sig") | Strips a leading BOM (U+FEFF). | Only the BOM at the start; not zero-width characters mid-text. |
To match the site's per-line trimming, iterate lines: "\n".join(line.strip() for line in s.splitlines()). str.splitlines() also splits on U+2028, U+2029, and other separators, which is usually what you want.
JavaScript
JavaScript follows a similar model. String.prototype.trim() removes whitespace and line terminators from both ends, and the regex \s class matches U+0020, tab, the Unicode space separators including U+00A0, and line terminators.
s.replace(/\s+/g, " ").trim()collapses runs to one space across the whole string, newlines included.s.replace(/[^\S\r\n]+/g, " ")collapses horizontal whitespace while keeping line breaks.s.replace(/[\u200B\u2060\uFEFF]/g, "")removes the zero-width space, word joiner, and BOM. Add\u200C\u200Dto the class only when the text has no joined scripts or emoji.s.normalize("NFKC")folds many compatibility spaces, including U+00A0 and U+2007, to a regular space as a side effect. Use it deliberately, because it also changes other characters.
Command line
Shell tools operate on bytes unless the locale is set, so multi-byte Unicode spaces are the main hazard.
sed 's/[[:space:]]\{1,\}/ /g'collapses ASCII whitespace runs; it will not see U+00A0 as space in the C locale.tr -s '[:space:]' ' 'squeezes runs but also flattens newlines.sed 's/[ \t]*$//'trims trailing spaces and tabs per line, a common pre-commit cleanup.- For Unicode spaces, use
perl -CSD -pe 's/\s+/ /g'or a Python one-liner, which decode UTF-8 first.
Spreadsheets
Excel and Google Sheets TRIM is the most common source of confusion. It removes leading, trailing, and repeated U+0020 spaces only. It does not touch a non-breaking space (character 160), which is exactly the character that arrives when data is pasted from a web page. The fix is to substitute it first:
=TRIM(SUBSTITUTE(A1, CHAR(160), " "))
Also remove a zero-width space:
=TRIM(SUBSTITUTE(SUBSTITUTE(A1, CHAR(160), " "), UNICHAR(8203), ""))For a whole sheet this is tedious, which is a case where pasting the affected column into the NBSP tool or the space remover and pasting it back is faster than building nested formulas.
Side-by-side comparison
| Need | Browser tool | Python / JS | sed / tr | Spreadsheet |
|---|---|---|---|---|
| Collapse multiple spaces | Yes, one step | Yes | Yes (ASCII) | TRIM |
| Remove all spaces | Yes | Yes | Yes | SUBSTITUTE |
| Handle NBSP and Unicode spaces | Yes, detected and counted | Yes with the right pattern | Only with perl or a UTF-8 locale | Only via SUBSTITUTE(CHAR(160)) |
| Remove zero-width characters | Yes, with a cautious mode | Yes with an explicit class | Hard | SUBSTITUTE(UNICHAR(8203)) |
| Per-line trim, keep line breaks | Yes | Yes with a loop | Yes with an anchor | Per cell only |
| Repeatable on many files | No | Yes | Yes | Limited |
| No environment or install | Yes | No | Shell needed | App needed |
| Shows what was hidden | Yes, live audit | Only if you print it | No | No |
A short decision guide
- One block of text, or a value you need to inspect: use a tool here and read the audit.
- A column in a spreadsheet: substitute CHAR(160) then TRIM, or round-trip the column through a tool.
- Many files, a data pipeline, or a build step: write Python or JavaScript with an explicit whitespace class and test cases.
- A quick trailing-space cleanup in version control: a
sedor editor "trim trailing whitespace" setting. - Unsure which characters are involved: diagnose with a tool first, then encode that finding in code.
The definitions used here follow the WHATWG strip and collapse ASCII whitespace algorithm and Unicode Annex #14. For the full character list, see the whitespace character reference; for data-specific cases, see whitespace in data cleaning.