it's very likely indeed that you ticked the "Email" checkbox below the comment form - willingly or inadvertently. Granted, the email title is a bit misleading, as you would receive a notification for every new comment in the thread, not just those from people who specifically answer you (I don't do threaded comments on the site - yet). The chances for these to be spam are very low, as it would require the spammer to impersonate the email account of the system.
In any case, there really isn't much value in these emails beyond the notification purpose - it's quite all right to delete them anyway.
Please do let me know if you're positive you didn't tick the box and still receive notifications, as I would then investigate to see if something is, well... working when we don't expect it to (Hey, that would be a new one...)
we're not there yet, but I definitely consider any step in that direction, no matter how modest, to be a perfectly reasonable use of my kinetic energy.