Humaid AlQassimi

< Go back

Detecting the Empty Character in Go

Jul 29, 2020 · 2 min read

I have recently been working on an online ticketing system. I have been using strings.TrimSpace for a while, and it works well. I tested it with the “empty character” from, and it failed to detect whatever whitespace characters it was using.

I thought it was just strings.TrimSpace not detecting different types of Unicode’s empty characters. So I replaced it with strings.TrimFunc(s, unicode.IsSpace), and it still didn’t clear the spaces1.

Disecting that empty character, we find it actually made up of five different characters:

  • U+200F: Right-To-Left Mark
  • U+200F: Right-To-Left Mark
  • U+200E: Left-To-Right Mark
  • U+0020: Regular Space
  • U+200E: Left-To-Right Mark

We can see that it is using a control character to prevent the regular space from being trimmed.

However, Go doesn’t list these characters as control characters2, so we cannot use unicode.IsControl. But it is included in the unicode.Bidi_Control subset. Here’s my first solution:

func isImproperChar(r rune) bool {
	return unicode.IsSpace(r) || unicode.In(r, unicode.Bidi_Control)

strings.TrimFunc(s, IsImproperChar)

This would trim away at bi-directional control characters, which is probably a really bad idea especially in systems supporting Arabic, Hebrew, or other right-to-left languages.

So we can just trim it to measure the length, then discarding the trimmed result.

func IsEmpty(s string) bool {
	return len(strings.TrimFunc(s, func(r rune) bool {
		return unicode.IsSpace(r) || unicode.In(r, unicode.Bidi_Control)
	})) == 0

Try it out on the Go playground!

Have a better solution? Please let me know!

This is my eighth post in the #100DaysToOffload challenge.

  1. I thought unicode.IsSpace wasn’t detecting detecting some types of spaces. But after some testing, that doesn’t seem to be the case. ↩︎

  2. Not listed on unicode/tables.go:7108 as pC (control character), but rather it’s included in the Bidi_Control subset. ↩︎

Would like to comment on the blog post? Feel free to start a discussion on my public general mailing list.

Articles from blogs I follow around the net

You can also read my newsfeed.

Status update, December 2020

Happy holidays! I hope everyone’s having a great time staying at home and not spending any time with your families. It’s time for another summary of the month’s advances in FOSS development. Let’s get to it! One of my main focuses has been on sourcehut’s API…

via Drew DeVault's blog December 15, 2020

Become shell literate

Shell literacy is one of the most important skills you ought to possess as a programmer. The Unix shell is one of the most powerful ideas ever put to code, and should be second nature to you as a programmer. No other tool is nearly as effective at commanding…

via Drew DeVault's blog December 12, 2020

Flip Phone Challenge Complete

Flip Phone Challenge Complete Written By: Jake Bauer | Posted: 2020-12-10 | Last Updated: 2020-12-10 Last week I challenged myself to use a flip phone for one week straight. It turns out that it really wasn’t too hard to get rid of my Android phone and star…

via - What's New December 10, 2020

Generated by openring