They are scientifically a fruit because of there seeds.
People call them vegetables because of their savory taste.
People determine if something is savory or sweet because they can tell by their taste if it is sugary,or has just a sweet taste like chocolate or candies or even most of fruits but not all ;) (you know like tomatoes) but savory is salty, garlicky spicy or a lot of other tastes.