Hi Anjampie-
This is a pretty common question, especially for people who might have used WordPress.com-hosted sites in the past. That’s because this feature is built-in to sites hosted there, but not by default into self-hosted WordPress sites that don’t have the extra features Automattic adds to those websites.
I’d recommend checking out the Subscriptions module of the Jetpack plugin. You can use the free version of Jetpack for this feature. It adds an area where your visitors can subscribe to new posts as well as to new comments on posts.