Rogan said he would take his two sons to the Auschwitz concentration camp in Poland, where it is estimated that one-third of the Nazis' victims were Hungarian.
Some posters held by protesters mocked Gyongyosi by showing him with a Hitler moustache and the crowd chanted "Jobbik go away!"
Attila Mesterhazy said his opposition Socialist Party would boycott Parliament's foreign affairs committee as long as Gyongyosi remained its vice chairman. He also called on Prime Minister Viktor Orban to address the issue in Parliament next week.
Jobbik president Gabor Vona, however, said the protest was part of an "artificially induced campaign of lies" meant to divert attention away from Hungary's economic problems and that the groups that took part in the rally wanted to "destroy" Jobbik.