The internet button is on, but I don't get internet from the router, only directly from the modem.

This an wire setup gone wrong!
The wire you use to get the internet from on your modem, plugs into I believe a Yellow port at the back of your router.
The rest of the ports are for hardwiring each computer.
So your computer, We say #1, will have a wire from you LAN card at the back of you computer to port #1 on you router.
you will have to go into your windows internet setup to change some settings. So to do this
My computer, Control Panel, Internet setup.Internet wizard setup.
You no longer SAY you use modem for internet, but setup through router.
This will NOT be a new setup but an existing setup