Python / SQL — Remove all non ASCII characters from unicode string

Here’s a brute force method to force a unicode string to ASCII by removing all non ascii characters from a unicode string for those pesky Ordinal Not In Range errors or DB errors.

Though I found comprehensive solutions that replace the non ASCII characters and such with approximate values, I just wanted to get rid of them, ASAP, Yesterday.

''.join([x for x in 'YOUR_STRING' if ord(x) < 128])

Published by Yuji Tomita

I'm a human person. I excel in crafting holistic technology solutions that extend beyond a narrow technical scope.

5 Comments

  1. Pingback: Remove Non-ASCII Characters Using Python « MKTGR
  2. fgm2r says:
    April 25, 2011 at 11:41 am

    Hi, thanks for this simple method to get rid of non-ASCII characters. I’m using urllib.quote and it literally throws up when it sees a unicode string.
    I think you mean “if ord(x) < 128]" though.

    Reply
    1. Yuji says:
      April 25, 2011 at 11:45 am

      You’re absolutely right 😀

      Reply
  3. Steven Callister says:
    August 19, 2012 at 9:17 pm

    Thanks, worked great 🙂

    Reply
  4. Tahir says:
    May 23, 2018 at 12:16 am

    raw = ”.join([i if ord(i) < 128 else ' ' for i in data]) #if you want to replace non-ancii char with space

    Reply

Leave a Comment