When to use a tool and when to write code

Use a browser tool when the job is bounded: a block of text pasted once, a value checked before it goes into a form, a document cleaned before publishing. There is nothing to install, the live audit shows which hidden characters were present, and the transformation is deterministic. Use code when the job repeats: a folder of files, a column that arrives daily, a build step, or any process that another person or a server needs to run without you.

The two approaches also fail differently. A tool keeps you in the loop, so you notice when a cleanup joins two words. A script applies the same rule to a thousand rows whether or not that rule was right, so it needs test cases. Many people use both: the tool to work out which characters are involved, then a script that targets exactly those.

Python

Python 3 strings are Unicode, and the re module treats \s as Unicode whitespace by default, so patterns match NBSP and most typographic spaces. The string methods behave as follows:

ExpressionEffectWatch out for
s.strip()Removes leading and trailing whitespace, including Unicode spaces.Only the outer edges of the whole string; interior lines untouched.
" ".join(s.split())Collapses every run of whitespace to one space and trims ends.Also collapses newlines, so line structure is lost.
re.sub(r"\s+", " ", s)Collapses whitespace runs to one space.\s includes \n; use r"[^\S\n]+" to keep newlines.
s.replace("\u00A0", " ")Converts only NBSP to a regular space.Misses U+202F and U+2007; chain replacements or use a regex class.
"".join(s.split())Removes all whitespace entirely.Joins words and values with no separator.
s.encode().decode("utf-8-sig")Strips a leading BOM (U+FEFF).Only the BOM at the start; not zero-width characters mid-text.

To match the site's per-line trimming, iterate lines: "\n".join(line.strip() for line in s.splitlines()). str.splitlines() also splits on U+2028, U+2029, and other separators, which is usually what you want.

JavaScript

JavaScript follows a similar model. String.prototype.trim() removes whitespace and line terminators from both ends, and the regex \s class matches U+0020, tab, the Unicode space separators including U+00A0, and line terminators.

  • s.replace(/\s+/g, " ").trim() collapses runs to one space across the whole string, newlines included.
  • s.replace(/[^\S\r\n]+/g, " ") collapses horizontal whitespace while keeping line breaks.
  • s.replace(/[\u200B\u2060\uFEFF]/g, "") removes the zero-width space, word joiner, and BOM. Add \u200C\u200D to the class only when the text has no joined scripts or emoji.
  • s.normalize("NFKC") folds many compatibility spaces, including U+00A0 and U+2007, to a regular space as a side effect. Use it deliberately, because it also changes other characters.

Command line

Shell tools operate on bytes unless the locale is set, so multi-byte Unicode spaces are the main hazard.

  • sed 's/[[:space:]]\{1,\}/ /g' collapses ASCII whitespace runs; it will not see U+00A0 as space in the C locale.
  • tr -s '[:space:]' ' ' squeezes runs but also flattens newlines.
  • sed 's/[ \t]*$//' trims trailing spaces and tabs per line, a common pre-commit cleanup.
  • For Unicode spaces, use perl -CSD -pe 's/\s+/ /g' or a Python one-liner, which decode UTF-8 first.

Spreadsheets

Excel and Google Sheets TRIM is the most common source of confusion. It removes leading, trailing, and repeated U+0020 spaces only. It does not touch a non-breaking space (character 160), which is exactly the character that arrives when data is pasted from a web page. The fix is to substitute it first:

=TRIM(SUBSTITUTE(A1, CHAR(160), " ")) Also remove a zero-width space: =TRIM(SUBSTITUTE(SUBSTITUTE(A1, CHAR(160), " "), UNICHAR(8203), ""))

For a whole sheet this is tedious, which is a case where pasting the affected column into the NBSP tool or the space remover and pasting it back is faster than building nested formulas.

Side-by-side comparison

NeedBrowser toolPython / JSsed / trSpreadsheet
Collapse multiple spacesYes, one stepYesYes (ASCII)TRIM
Remove all spacesYesYesYesSUBSTITUTE
Handle NBSP and Unicode spacesYes, detected and countedYes with the right patternOnly with perl or a UTF-8 localeOnly via SUBSTITUTE(CHAR(160))
Remove zero-width charactersYes, with a cautious modeYes with an explicit classHardSUBSTITUTE(UNICHAR(8203))
Per-line trim, keep line breaksYesYes with a loopYes with an anchorPer cell only
Repeatable on many filesNoYesYesLimited
No environment or installYesNoShell neededApp needed
Shows what was hiddenYes, live auditOnly if you print itNoNo

A short decision guide

  1. One block of text, or a value you need to inspect: use a tool here and read the audit.
  2. A column in a spreadsheet: substitute CHAR(160) then TRIM, or round-trip the column through a tool.
  3. Many files, a data pipeline, or a build step: write Python or JavaScript with an explicit whitespace class and test cases.
  4. A quick trailing-space cleanup in version control: a sed or editor "trim trailing whitespace" setting.
  5. Unsure which characters are involved: diagnose with a tool first, then encode that finding in code.

The definitions used here follow the WHATWG strip and collapse ASCII whitespace algorithm and Unicode Annex #14. For the full character list, see the whitespace character reference; for data-specific cases, see whitespace in data cleaning.