operation on text

Case & character operations

  • Uppercase, lowercase, title case, sentence case, camelCase, PascalCase, snake_case, kebab-case
  • Character substitution (tr, sed s///)
  • Diacritic/accent stripping (normalize é→e)
  • Full-width ↔ half-width character conversion (CJK contexts)
  • Case folding for case-insensitive comparison

Whitespace & structure

  • Trim leading/trailing whitespace
  • Collapse multiple spaces/tabs into one
  • Line wrapping / unwrapping (fold, fmt)
  • Indentation conversion (tabs ↔ spaces, tab width change)
  • Line-ending conversion (CRLF ↔ LF, dos2unix/unix2dos)
  • Line sorting, deduplication, shuffling (sort, uniq, shuf)
  • Line numbering / stripping numbers
  • Column alignment (column, paste)
  • Joining lines / splitting on delimiter

Encoding & character sets

  • Character encoding conversion (UTF-8, UTF-16, Latin-1, ASCII) — iconv
  • Encoding detection (uchardet, chardet)
  • Base64 encode/decode
  • URL encode/decode (percent-encoding)
  • HTML entity encode/decode (& ↔ &)
  • Unicode normalization forms (NFC, NFD, NFKC, NFKD)
  • Escape/unescape sequences (\n, \t, \", regex-escape)
  • Hex ↔ text, binary ↔ text, ROT13, Caesar cipher (novelty/education)

Format & markup conversion

  • Markdown ↔ HTML ↔ LaTeX ↔ DOCX ↔ ODT (pandoc)
  • Plaintext ↔ rich text
  • CSV ↔ JSON ↔ YAML ↔ XML ↔ TOML
  • Table reformatting (Markdown table ↔ CSV ↔ ASCII table)
  • Syntax highlighting injection/removal
  • Front matter extraction (YAML headers in Markdown)

Structural transformation

  • Find and replace (literal, regex, multiline)
  • Text extraction (regex capture groups, XPath, JSONPath)
  • Templating/interpolation (Jinja2, mustache)
  • Merging multiple files, splitting one file into many
  • Diffing and patching (diff, patch)
  • Deduplication of paragraphs/blocks
  • Reordering/sorting by custom key
  • Column extraction/reordering (cut, awk)
  • Pivoting/reshaping tabular text (wide ↔ long)

Compression & packaging (text-adjacent)

  • Gzip/bzip2/xz text compression
  • Text file archiving into single bundle (tar, zip of .txt files)

Linguistic/NLP operations

  • Tokenization (word, sentence, subword/BPE)
  • Stemming, lemmatization
  • Stopword removal
  • Part-of-speech tagging
  • Named entity extraction
  • Transliteration (script ↔ script, e.g., Tamil ↔ Latin, Devanagari ↔ Latin)
  • Translation (language ↔ language)
  • Spell-check and correction
  • Word/character/line counting (wc)
  • Readability scoring
  • N-gram extraction
  • Text summarization
  • Paraphrasing
  • Sentiment/keyword extraction

Validation & analysis

  • Syntax validation (JSON lint, YAML lint, XML well-formedness)
  • Schema validation (JSON Schema)
  • Regex matching/testing
  • Duplicate line/content detection
  • Checksum/hash generation (md5sum, sha256sum) for integrity, not content
  • Language detection

Security/privacy-related text ops

  • PII redaction/masking
  • Encryption/decryption (gpg on text files)
  • Digital signing of text content

Automation/pipelining ops

  • Batch renaming based on text content
  • Watch-and-transform (inotify + script triggers)
  • Templated document generation (mail-merge style)
  • Scripted multi-step pipelines chaining several of the above (e.g., OCR → clean → translate → format)

If you want, I can convert this into a lookup table matching operation → specific CLI tool → Fedora/Termux install command, similar to your other reference-list formats.


You'll only receive email when they publish something new.

More from பிரசாந்த்
All posts