The Jews were often seen as a lesser "race" even though they weren't any different than the people around them, they have also been blames for countless things throughout time
The Nazi's would put the Jews into ghettos. Where they would have no way of getting information from the outside. Then they would load them on busses, lead them to camps and either kill them on the spot or work them till they died
It is very hard to determine who should be held responsible besides Hitler. The best way would be to set up a set of requirements and if you meet or don't meet those that would determine if you were guilty or not.
A lot of them attempted to leave, but German laws made it almost impossible with multi step processes, and even if they got passed that the country they were going to had more difficult tasks to complete before they could come in