Humaid AlQassimi

< Go back

Detecting the Empty Character in Go

Jul 29, 2020 · 2 min read

I have recently been working on an online ticketing system. I have been using strings.TrimSpace for a while, and it works well. I tested it with the “empty character” from, and it failed to detect whatever whitespace characters it was using.

I thought it was just strings.TrimSpace not detecting different types of Unicode’s empty characters. So I replaced it with strings.TrimFunc(s, unicode.IsSpace), and it still didn’t clear the spaces1.

Disecting that empty character, we find it actually made up of five different characters:

  • U+200F: Right-To-Left Mark
  • U+200F: Right-To-Left Mark
  • U+200E: Left-To-Right Mark
  • U+0020: Regular Space
  • U+200E: Left-To-Right Mark

We can see that it is using a control character to prevent the regular space from being trimmed.

However, Go doesn’t list these characters as control characters2, so we cannot use unicode.IsControl. But it is included in the unicode.Bidi_Control subset. Here’s my first solution:

func isImproperChar(r rune) bool {
	return unicode.IsSpace(r) || unicode.In(r, unicode.Bidi_Control)

strings.TrimFunc(s, IsImproperChar)

This would trim away at bi-directional control characters, which is probably a really bad idea especially in systems supporting Arabic, Hebrew, or other right-to-left languages.

So we can just trim it to measure the length, then discarding the trimmed result.

func IsEmpty(s string) bool {
	return len(strings.TrimFunc(s, func(r rune) bool {
		return unicode.IsSpace(r) || unicode.In(r, unicode.Bidi_Control)
	})) == 0

Try it out on the Go playground!

Have a better solution? Please let me know!

This is my eighth post in the #100DaysToOffload challenge.

  1. I thought unicode.IsSpace wasn’t detecting detecting some types of spaces. But after some testing, that doesn’t seem to be the case. ↩︎

  2. Not listed on unicode/tables.go:7108 as pC (control character), but rather it’s included in the Bidi_Control subset. ↩︎

Would like to comment on the blog post? Feel free to start a discussion on my public general mailing list.

Articles from blogs I follow around the net

You can also read my newsfeed.

A few ways to make money in FOSS

I work on free and open-source software full time, and I make a comfortable living doing it. And I don’t half-ass it: 100% of my code is free and open-source. There’s no proprietary add-ons, no periodic code dumps, just 100% bona-fide free and open source so…

via Drew DeVault's blog November 20, 2020

We can do better than DuckDuckGo

DuckDuckGo is one of the long-time darlings of the technophile’s pro-privacy recommendations, and in fact the search engine that I use myself on the daily. They certainly present a more compelling option than many of the incumbents, like Google or Bing. Even…

via Drew DeVault's blog November 17, 2020

Gemini is Up and Running

Gemini is Up and Running Written By: Jake Bauer | Posted: 2020-11-02 | Last Updated: 2020-11-02 This post is also available on Gemini. The Gemini protocol is a new “small Internet” protocol which aims to sit in between Gopher and the HTTP-based Web in terms …

via - What's New November 2, 2020

Generated by openring