Yes, if you'd provide constant push to the Sun, you'd get the junk to fall down your way. Otherwise the junk would assume ellyptical orbit as soon as you stop pushing and would become another huge dangerous asteroid swirling in the close proximity of Earth.

Sun's gravity would gladly take care of that, as you say, if there wasn't a little itsy bitsy problem. Earth is notstationary. Anything we launch from Earth has 30 kilometers per second of velocity from the beginning, and that velocity is not in the right direction. To put it simply - you ain't going anywhere just by pushing the junk away - it will remain in near-Earth orbit and that's it.

The only thing you can do is to launch the junk into the opposite direction from where Earth is traveling. In other words - slow it down. Considerably, by the way. 30 km/s is not that little.

I will write more as soon as I get more data and, preferably, some illustrations for you to understand.