Login
You're viewing the mastodonapp.uk public feed.
  • Graham Sutherland🎃 / Polynomial.pdf.exegsuberland@chaos.social
    Oct 26, 2026, 4:49 PM

    @fluepke Japanese characters can be written in wide or narrow form, which have different Unicode encodings. the width depends on the device you're typing on, with narrow being common as a default on mobile soft-keyboards and wide being the typical default when writing on a desktop keyboard. as such, passwords written in CJK scripts may encode to different byte sequences depending on whether the user logs in via mobile or desktop, and therefore produce different hashes/keys.

    💬 4🔄 11⭐ 18

Replies

  • Graham Sutherland🎃 / Polynomial.pdf.exegsuberland@chaos.social
    Oct 26, 2026, 4:53 PM

    @fluepke you have to run NFC normalisation over the string first to get a consistent byte sequence.

    however! this is not actually sufficient on its own. another gotcha is that the normalisation result depends on the Unicode version. for long term stability (e.g. if you're using the password as an input to a KDF for bulk storage encryption) you need to pin the Unicode version alongside the hash. failing to do so many result in a user no longer being able to recover data.

    💬 1🔄 1⭐ 8
  • Graham Sutherland🎃 / Polynomial.pdf.exegsuberland@chaos.social
    Oct 26, 2026, 4:57 PM

    @fluepke and since a new user password might contain grapheme clusters that only exist in newer Unicode versions, you either need to have a version upgrade path (no global version pin) or reject any password that contains an unrecognised cluster.

    this is even more difficult if you're doing any kind of client-side / server-side agreement, especially across distinct platform and language stacks, because both sides need to be on the same Unicode version and be doing the same normalisation.

    💬 1🔄 0⭐ 6
  • 💬 3🔄 2⭐ 6
  • 💬 1🔄 0⭐ 0
  • 💬 1🔄 1⭐ 3
  • 💬 0🔄 0⭐ 1
  • Graham Sutherland🎃 / Polynomial.pdf.exegsuberland@chaos.social
    Oct 26, 2026, 5:15 PM

    @fluepke also also: if you're doing any kind of nonrepudiation of string equality where malicious modification is in scope you also need to authenticate the Unicode version that was used, e.g. in the AD portion of an AEAD.

    💬 1🔄 1⭐ 2
  • Flüpkefluepke@chaos.social
    Oct 26, 2026, 5:19 PM

    @gsuberland maybe we should go back to opaque octet strings, that shall carry no particular meaning at all.

    But that doesn't really solve any of the issues either, admittedly.

    💬 0🔄 1⭐ 3
  • @dingensdingens@troet.cafe
    Oct 26, 2026, 10:44 PM

    @gsuberland @fluepke what's the threat scenario? I don't see it: the input is from the user, the normalization algorithm isn't secret, so all good, isn't it? (Assuming the DB contains `hash(normalize(user_pw))` )

    💬 0🔄 0⭐ 0
  • Flüpkefluepke@chaos.social
    Oct 26, 2026, 5:17 PM

    @gsuberland is this the similar to the U+0308 + U+006F ("+o) vs U+00F6 (ö) normalization in German?

    TIL that MacOS uses the ugly combining diaeresis variant.

    💬 3🔄 0⭐ 3
  • Hraban (fiëé visuëlle)fiee@literatur.social
    Oct 26, 2026, 6:41 PM

    @fluepke
    It depends on the font and the Unicode renderer if it’s ugly (often you can’t see the difference), but decomposed Unicode is a problem for many applications and web forms (e.g. German Post for a long time didn’t accept addresses with umlauts that I copied from a PDF on MacOS; I kept complaining and explaining, and at some point in history they fixed it).

    @gsuberland

    💬 0🔄 1⭐ 0
  • 💬 1🔄 0⭐ 0
  • Graham Sutherland🎃 / Polynomial.pdf.exegsuberland@chaos.social
    Oct 26, 2026, 8:51 PM

    @fluepke the length is also wonky as hell. Swift uses extended grapheme clusters, Python/Ruby use unicode code points, Java/JS/TS/C#/Obj-C use UTF-16 code units, Rust/Go/PHP generally use byte length in UTF-8 encoding. breaks all sorts of things, cryptographic and otherwise.

    also things like character class checks are totally different across languages.

    💬 2🔄 1⭐ 0
  • Graham Sutherland🎃 / Polynomial.pdf.exegsuberland@chaos.social
    Oct 26, 2026, 8:53 PM

    @fluepke my favourite example of this was a platform with web/android/iOS apps, where signing up with a password containing diacritics or emojis would let you log in, but if you switched apps it wouldn't let you log in because the complexity check got re-ran on login (it was a shared component). for some passwords you could have one app approve the length, but another reject it. for other passwords you could have one app approve the complexity requirements, but another rejected it. wacky.

    💬 0🔄 0⭐ 0
  • 💬 1🔄 0⭐ 0
  • 💬 1🔄 0⭐ 0
  • nadjadequbed@mastodon.chaosfield.at
    Oct 26, 2026, 8:59 PM

    @gsuberland @fluepke yeah, I figured. By the time some input touches cryptography it really ought to be some opaque array of bytes you can assume nothing about or you *will* give yourself an exploit-shaped headache :[

    💬 0🔄 1⭐ 1
  • s3lphs3lph@chaos.social
    Oct 26, 2026, 9:15 PM

    @fluepke @gsuberland while migrating mail homes between different IMAP servers, I encountered some users who had TWO draft folders. Both called "Entwürfe", but one with ü, the other with "+u (which Apple Mail produces). That was fun to figure out, espectially because migration was failing with a "folder already exists" error from the destination server.

    Still gotta upstream that imapsync patch at some point...

    💬 0🔄 1⭐ 0
  • philbieberium@infosec.exchange
    Oct 27, 2026, 8:17 AM

    @gsuberland @fluepke A certain Japanese airline (and I bet it is not limited to them)even has the PIN requirement of "4 half-width numbers".
    It struck me as odd when seeing this, and now I don't want to contemplate the implications...

    💬 1🔄 2⭐ 0
  • 💬 0🔄 0⭐ 0
  • 💬 2🔄 0⭐ 0
  • 💬 0🔄 1⭐ 0
  • Emelia/Emibecomethewaifu@tech.lgbt
    Oct 27, 2026, 12:11 PM

    @alice @gsuberland @fluepke Yeah, my input methods only support half-width for katakana. Compressing all but the simplest kanji like that would render them "rather illegible." There are multiple ways to encode korean though.

    Since hangul uses "multiple sub-characters per character" (oversimplified, but that's the gist) there's two ways to represent it in unicode: "pre-composed" codepoints representing an entire syllable, or as individual decomposed elements. Both display as the same glyphs.

    That said, unicode normalization will convert everything to the latter form, so it shouldn't be that big of a deal in practice? Assuming whoever is programming something that needs it is competent enough to understand that they need it in the first place...

    💬 0🔄 1⭐ 2
  • 💬 0🔄 0⭐ 0