Why is YubiHSM considered an HSM?

Size and performance don't matter, as a hardware security module (HSM) is defined by its functions to perform cryptographic operations and protection. From Peter Smirnoff on Cryptomathic: Understanding Hardware Security Modules (HSMs): The hardware security module (HSM) is a special “trusted” network computer performing a variety of cryptographic operations: key management, key exchange, encryption etc. It seems to be obvious that cryptographic operations must be performed in a trusted environment. When I say trusted, I mean “no viruses, no malware, no exploit, no unauthorized access.” An HSM is trusted because it: Is built on top of specialized hardware. The hardware is well-tested and certified in special laboratories.

Has a security-focused OS. Has limited access via a network interface that is strictly controlled by internal rules. Actively hides and protects cryptographic material. You can find the same list of the key properties of a HSM e.g. from Doron Gez: What Is a Hardware Security Module. Although YubiHSM is nano form factor, it meets these requirements (YubiHSM 2 Product Overview). While bigger appliances could do these operations faster and store more keys, that only affects their possible use cases.