To find out how does Google know my location, you should know that Google uses BSSID information from your WLAN Access Point to get an approximation of where you are located, even with GPS and WiFi turned off. Taken from “How does Google Maps estimate my location without GPS?”:
Google and others like Apple and Skyhook build a Database which links WLAN BSSIDs to a geographic location. A BSSID is like the MAC address of an access point that gets broadcasted by that access point. It is therefore "public viewable" if the BSSID broadcast is enabled, which is the default for most access points. The BSSID operates on a lower layer as the IP stack, you don't even have to be connected to an access point to receive these broadcasts.
So, essentially, when you ARE using WiFi and GPS, Google's database of BSSIDs is updated with a geographic location associated with that BSSID, as you've assumed. In your case, your AP is sending beacons advertising its BSSID, and because it is already in Google's database, Google Maps knows where you are based on the location of that AP.
So it's not that the ISP is giving Google the location of their routers, it's that your phone has already helped to build a database of the Access Points around you, and Google uses this data for geolocation.
Sadly, even if you get a new router and keep any and all Android devices away from it, they will still be able to approximate your location based on the cell towers your phone connects with (or maybe even your neighbor's AP!), but it won't be nearly as accurate.
I saw in the comments questions about whether or not Android phones will receive location data even with WiFi turned OFF. The answer is, yes, absolutely they can. I'm sorry I didn't make that clearer. Better check your settings if you were unaware:
The Advanced Wi-Fi settings menu for Android devices. The setting “Scanning always available” is underlined in red for emphasis. The setting's toggle switch is in the On position. The setting's in-menu explanation says “Let Google's location service and other apps scan for networks, even when Wi-Fi is off”.
This "feature" has been included since Android 4.3, and prior versions of the Android OS do not include this feature.
Although turning off this "feature" on your phone seems like the best way to prevent your BSSID from being added to the database, this isn't necessarily the case. You've got other people's phones, the phones of passers-by, and even Google's own Street View cars to contend with.
Though this may be the case, you can opt out of your involvement in this program by appending _nomap to the end of your SSID. Your SSID is the "name" of the network that you have chosen or have been given. For example, you connect to the SSID
"Home" or "D-Link" for your WiFi at home. In order to opt out you would rename your network Home_nomap or D-Link_nomap.<br>
