Usually anti jokes are funny due to a lack of punchline, but this joke does actually have a functioning punchline and subversion of expectation. It just doesnt work as well if you dont get that the almond bit is about poison.