I don't know a whole lot, but I think the reason the show went downhill at the beginning of Johto was that the fad died. Pokemon was BIG in Kanto days and even into the Orange Islands saga, but a fad can't last forever and like all fads it came and went, not completly or pokemon would still not be here.
Now at the Advanced series Gen III, they started fresh, tried to do things all over again. I liked Hoenn, from what I did see of it. This might have been why Misty went and was replaced my May and Max.
Ash did not have his team this time when he first appeared in Hoenn, another thing to show that they "started fresh" more so for a new geration of Kids. After the fad died and fans started losing interest, they had to do something. Just having Ash with Pikachu would let fans see how Ash would go about capturing pokemon and challenging the gyms there, something that was ported onto D/P series. Probably why Brock was never there at the start of both sagas either, just Ash and Pikachu.
If you were a new fan to the show and suddenly saw Ash in the new region with pokemon like Charizard/Noctowl etc... what would they think? They'd feel "out of the loop" if you will and might not take to the show.
Older fans complain that older pokemon get "forgotten" when they are Oaked, but they make a return at some point, its like older characters too like Misty or Tracy etc... people complain they are not making any returns either, this is why, the new generation of kids watching the show don't wan't to feel like they mised out something, and anyway there are references to past pokemon, sutuations and people and these are done in such a way that makes sense:
Ash is in a situation like "Dustox crossing" episode or something and this brings back memories of butterfree for Ash, and then Ash would explain o sayawn who did not know Ash then or about his previous non Sinnoh pokemon, how he had a similar situation when he caught a caterpie and it evolved into butterfree and he had to let it go with a pink female butterfree to have eggs and more butterfree.
There is always a rival for Ash, plus the main rival he always had, Gary. The rival is there to push Ash and make him do his best with his pokemon, the way he knows how, like how Paul has been with Ash this series, pushing Ash to be better and better and stronger so he can beat him and prove his strength and this builds character.
Now we know the games and the anime go hand in hand, the anime advertises the games. Ash ahs started with only pikachu in Hoenn and Sinnoh, now this is similar to how your player is in the current game for that region. You start by getting a starter, your only pokemon (anime has Ash and Pikachu) you meet your rival (sort of like how Ash meets female in anime, but he goes with them) then you start catching/training pokemon and challenge gyms and league with your pokedex and items, (what Ash does, catches pokemon from that region and uses them against gyms etc...) so you can see how it goes.
This is what Gen III did, started things this way for the reasons I mentioned above, and its stuck, as it works. Older fans might feel that some things get forgotten (GS ball, Misty, Jigglypuff etc...) but when has the anime been for older fans only?
Kanto had something other series didn't. It was the starting point and this was to test things to see how people would react to it. Like catching pokemon (maybe why a green pokeball was seen as games did not have colour in them days) or animals in anime that were gone in later shows also there was something that Indio league shows had that no other shows did, now I can't put my finger on it, maybe it was because the gang was not as mature as they appear now or that the storyline has become more serious, but its something, there was a lot more fillers in them days "Island of the giant pokemon" "Flame Poke-a-thalon" "Beauty and the beach" etc...
