The most unique sound in English is the "r".  It's the one that speakers of other languages just can't get exactly right. 

To me, the 'r' and the flap t in American English are among the distinguishing factors of English. I like to mention 'l' as well, especially at the end of the words, and I think, just in American English. In an episode where Martha and Grant were talking about some (I think, old-fashioned) 'names', they mentioned 'nyargle'. I would hear it as nyargoe until I looked it up. This is more tricky in Estuary English, where 'milk' is pronounced as mewk! 

The voiced and unvoiced th also happen in Arabic.

English doesn't have:

• The vowel sound spelled y in French and ü in German
• A true dental (tip of tongue against upper teeth) t or d sound, as opposed to the alveolar (tongue against the gums behind the teeth)
• A labiodental nasal, like the sound m but made between the upper teeth and lower lip rather than both lips together
• Any of the "click" sounds found in languages like Xhosa

(Interestingly, I've found the first three of those four useful in my Japanese pronunciation which my first teacher described as "almost native".)

There's a combination of sounds that I find difficult to make in Japanese, which is the liquid cluster at the beginning of the syllables rya, ryu and ryo. The individual sounds (even those that are not exactly duplicated in English speech) come easy, it's just putting them up against one another that's a struggle.

I think the schwa before 'r' tends to be somewhat longer, and dictionaries have different styles to show it: /'mɜːrdɜːr/ or /'mɚdɚ/. I get this by matching the British pronunciation of words as shown in IPA and pronounced by a British speaker. British dictionaries show /'mɜːdə(r)/. But the problem is that some dictionaries show it as /'mɜːrdɚ/. It's a long time I have this problem. I don't think they notice it as precisely as we do. Or the pronunciations are related to  delicate somewhat regional nuances- accents other than General American.

Ron Draney said
English doesn't have:

• A labiodental nasal, like the sound m but made between the upper teeth and lower lip rather than both lips together

Actually, English does have a labiodental nasal, in words like emphatic and anvil. Almost all languages assimilate a nasal to the point of articulation of a following consonant. What happens is that no language has been conclusively proven to have a phonemic labiodental nasal, distinguished from both bilabial and alveolar nasals.

I'd say that the most unique sound of English is the /ɹ/. It's not an alveolar approximant, like the symbol implies; at least in my dialect, it's a rounded retroflexed lateralized velar approximant, that in IPA would look like [ɰ˞ʷ].

RobertB said
Ron, what foreign words have a leading consonant m that is 'made between the upper teeth and lower lip' ?
Aldamans, could you give some examples of /ɹ/ ?

/ɹ/ is the English r-sound.