Over the span of the eight films, there is no way Harry would realistically stay with the Dursleys. If their abusive nature weren't enough to get the attention of wizards, we muggle-folk would handle that quickly before the end of the first film.
There is no consequence. Minimal reaction from magic folk. Seriously, is Harry somehow safer with the Dursleys? He is often faced with dark magic outside of school grounds, so realistically, living with a wizarding foster family would be the obvious way to handle this situation. It makes zero sense, and, as I will say with many on this list if the books explain it better, this is a list about the films.