I would like to have a site specific profile image/avatar, not something generic across the internet.

This article, from a WordPress perspective but generally applicable, discusses why local avatars are better for a variety of reasons: https://wptavern.com/local-avatars-in-wordpress-yes-please