Connection speed and 2 routers

Recently I got a battery doorbell with homebase. I noticed it takes a long time between a visitor pressing the bell and me getting a connection on my iPhone. And that was the mean reason for me to buy the doorbell. I saw several post on this forum about this issue.

There might be a complicating issue in my configuration. I have two routers, One and Two. Router One was my first router. Router Two is from my TV and Internet provider. On installation I asked the Wifi router Two to be turned off. My homebase is connected to router Two. Router One is connected by ethernet cable to router Two. The test on the homebase failed twice.

Is this configuration a problem?