Decided to dump the Xfinity cable router, for a router only since I'm Cat5e hardwired throughout the house. This combo had 4 ports on the back and hooked up to two different routers at opposite ends of the house and had the router that was part of the modem that was in the Smart Center where all the wires were. Worked fine other than the modem/router in the faraday cage had poor speed.

Picked up the CM2500 to replace the Xfinity CR unit and got it going, all was well until I tried to plug in the use the Link Aggregation, not sure what it is, as a second connection to a Cat5e port to run the 2nd router. Ever since I can only connect one router to the internet. Adding a Netgear Gs108 unmanaged switch did not help.

I dumped the Modem/Router combo because it sat in a metal box where the wires all are and had crappy speed because of the faraday cage it was in.

What am I missing?

Thanks!

Not completely sure your configuration, but looks wrong on multiple levels. First of all there should not be any number of routers greater than one. May be acceptable to have more than one but any additional must be in pass through or some access point mode. And if so they typically have to have things like their DHCP server disabled. Also link aggregation has to be configured on each end of the link, and is there really any reason you need need it at all. If you have internet service that provides greater than 1Gb service maybe, but today you would likely be better served with device that supports 2.5G ports.
If you really have two routers that have NAT in play my best guess is they are issuing the same address block and that is never going to work. I am totally down with splitting the modem/router device functionality, I never recommend combination devices.
Maybe a better picture of your network/device connections how to AFU-- your network.

What I had before was routers connected to the LAN ports via the hardwire in the house. I guess the question is why I can't do the same thing with only the one LAN port the CM2500 has, or how to split the port I have?

Most of these threads need a quick terminology check to make sure everyone is on the same page. The CM2500 is a cable modem, with no built-in routing capability. The link aggregation port is only meant to increase speeds to another single LAG-capable device. You can't use the second port to connect to a second device.

You also mention "routers" but it sounds like you may be talking about wireless access points (or at least a combo router / WAP device). What device in your network is performing NAT? The CM2500 won't do it, so if the two routers you mention are truly routers that perform NAT, then yes, it's no surprise you're seeing issues because only the one plugged into the primary (non-LAG) port on the CM2500 modem will pull a public IP address that then needs to be NAT'ed for devices on your local network. A switch between the CM2500 and any downstream routers won't cut it either, since only one public IP will be allocated and a dumb switch doesn't perform any functionality at the IP layer. You need one NAT-enabled router connected to the modem, then hang a switch off of the router and connect up all of your other gear (wired devices and wireless access points, if needed).

Edit: To answer your last question, your setup worked previously because the Xfinity device you replaced sounds like it was a combo modem/router. So it was pulling a single public IP address, then NATing that and the four LAN ports were effectively sitting on a switch behind the built-in router. If you had more downstream routers it would still work but was probably in a somewhat wonky config (double NATed). All you needed downstream from the old setup were downstream switches if you had multiple devices at each end of the house to connect.

So I hooked up one of the old wireless router's to the CM2500, used the Lan ports on the back of it to distribute to the ports in the two rooms where the other routers are. I'll just leave it with the Modem and probably turn off its wifi. It's an old Router but has enough LAN speed for the signal. What the CM2500 replaced was a modem/router combo that was in a metal case where the house cat5e wiring is hubbed. Since the case messed up the wifi signal there, I thought I would leverage the existing wiring.

Next up is to make the router in Room 2 a bridge of Room 1, just via the hardwire vs wirelessly to have the same SSID throughout the house.

Or should I not?

The goal is for a seamless wifi experience using what Routers I already have vs setting up a mesh.