10G Networking for Dummies

timeshifter

timeshifter

Well-Known Member
Reaction score
2,249
Location
USA
Have a small business client, they build customized machinery and the like. 3-4 of their users rely heavily on programs like AutoDesk Inventor. The owner wants to improve load times (and other operations) on his workstation. Based on some experimenting we've done we've decided to upgrade his machine and the server to 10G networking.

I'm in the process of moving the whole network under the UniFi umbrella. Just install a USG Pro, Cloud Key Gen 2, (2) UniFi 6 Lite APs and (1) UniFi FlexHD AP today. They have several switches that I'd like to upgrade at some point in the future too, all 1Gb switches.

So, for this upgrade I'd really really prefer to use a UniFi 10Gb switch.

The only switches that I see in the UniFi line that do 10Gb are these two:
store.ui.com

Switch XG 16 - Ubiquiti Store United States

Layer 2 aggregation switch with (12) 10G SFP+ ports and (4) 10GbE RJ45 ports.
store.ui.com store.ui.com
store.ui.com

Switch XG 6 PoE - Ubiquiti Store

A Layer 3, PoE switch with (4) 10 GbE, PoE++ RJ45 ports and (2) 10G SFP+ ports.
store.ui.com store.ui.com

They're both $599. The 6 port one is PoE, but don't really need that and it's out of stock. So I guess that leaves the 16XG?

Would also need NICs which UniFi doesn't make or sell, so I'd need to find what's good there.

Their wiring is ready, CAT6 runs that have been certified.

So I'd be ordering a switch, a couple of NICs and a few patch cords. Am I on the right track?
 
That desktop better be using an NVME SSD, because if it isn't... this is a waste of time. SATA is gigabit, you can't go faster than the slowest link.

As for Unifi switches yes that's all the 10gbit except for the US-48 Gen 1, that has two SFP+s on it, and I think the gen 2 24 port has a pair of SFP+s on it too.
 
timeshifter said:
It's got two of them along with an Intel Xeon Gold 6138 CPU @ 2.00 GHz, 64GB RAM and an NVidia Quadro P4000.
Click to expand...
And the server's drive array?

Though if the desktop is at that level... probably fine too.

The Unifi 16xg is what I've used for backbone work... as for NICs, I get stuff with Intel's name on it.
 
timeshifter said:
Are you sure? 1 Gbps = 125 MB/s. That SSD has a quoted read speed of 550 MB/s. Which would work out to about 4 Gbps? Not 10, but more than 1?
Click to expand...
You're not wrong, but I don't generally see 10gbit being worth it until I have at least a raid 10 made out of those things.
 
timeshifter said:
So what kind of read speed could we expect with a RAID 10 setup of those drives?
Click to expand...
About 80% of 2 times their rated speed. That's after the host OS loads and what not, so might be less if things get busy.

I just know that if I stuff a 10gbit card in a server that's got four of those running RAID 10, I can actually use that 10gbit link. I've never pushed all that to a single station though, that was a file server to support 80 people with gigabit access.
 
Research settings of your NICs...ensure you have good NICs...I also prefer Intel....and not the cheap ones (not "win" NICs...but good hardware controller based NICs).

Check settings on offloading. Test settings regarding jumbo frames, MTU, etc.
 
Which NIC to pick?

ASUS XG-C100C 10G Network Adapter Pci-E X4 Card with Single RJ-45 Port

Amazon.com: ASUS XG-C100C 10G Network Adapter Pci-E X4 Card with Single RJ-45 Port: Computers & Accessories

Buy ASUS XG-C100C 10G Network Adapter Pci-E X4 Card with Single RJ-45 Port: Network Cards - Amazon.com ✓ FREE DELIVERY possible on eligible purchases
www.amazon.com

Intel Ethernet Converged Network Adapter X540-T1 - network adapter - PCIe 2
www.cdw.com

Intel Ethernet Converged Network Adapter X540-T1 - network adapter - PCIe 2 - X540T1BLK - Ethernet Adapters - CDW.com

Buy a Intel Ethernet Converged Network Adapter X540-T1 - network adapter - PCIe 2 at CDW.com
www.cdw.com www.cdw.com

Others?
 
I stick with Intel when looking for top shelf networking cards. Just make sure and document everything along the communications paths. All the advertised speeds thing remind back in the dialup days when everyone got so excited when 56k modems came out and there weren't huge changes in performance.
 
We put the Gen1 48-port Ubiquiti switch in for a client back in August. Bonded 2x10G ports from the server (Intel NIC), which does have a RAID10 of SSDs. It's overkill, but it absolutely helped throughput to the workstations. They constantly pull X-Ray images from the server to display on workstations, and their practice-management app works much better since, as well.

We looked at the cost difference to a less-overkill solution, and figured it was $2K well-spent to never have to think about that problem again.
 
Yeah, technically we're in that grey space between 1gbit and 10gbit in terms of bandwidth.

And that's why we have 2.5GBASE-T and 5GBASE-T gear on the shelf, but very... VERY few switching options.

For business use though, I just go straight to 10GBASE-T solutions, costs a bit more but not much when compared to the less expensive alternatives and while we might not be able to make full use of it today, we will sometime in the next 3-5 years.

And that means I only really support 10gbit over FIBER, the only copper solution I'll muck with is a DAC cable. 10gbit over copper requires excessively tight tolerances, and it's one of those things that breaks when the air conditioner blows on a cable. I don't have time for that!

That means, the Intel X520-SR1 at a minimum. https://www.amazon.com/Intel-E10G41BFSR-Ethernet-Adapter-X520-SR1/dp/B002IYDH2Y
 
Last edited:
Sky-Knight said:
Yeah, technically we're in that grey space between 1gbit and 10gbit in terms of bandwidth.
Click to expand...

Right? I think it's overdue for the average computer to have more than a 1Gbps NIC. Something has to push the development of the infrastructure. It's still too expensive to get full 10Gb switches for the types of installs we do at the small end of the market. 10Gb NICs are still too expensive as well. I'm ready for the shift!
 
This project stalled at the end of the year. Picking it back up. Client understands that it would only be about 2.5 Gbps instead of 10 because of the server bottleneck. This is more of a proof of concept. If it works we'll end up with about 5 users on the 10G network. And we'll likely upgrade the server at that point.

Since I posted it looks like there's a new option
store.ui.com

Switch Aggregation - Ubiquiti Store

An 8-port, Layer 2 switch made for 10G SFP+ connections.
store.ui.com store.ui.com

About 1/3 the price. But, I like the 16 XG better for the flexibility.
 
Just checking the basics when you mentioned using CAT6

CAT6 - maximum distance of 55m at 10GB
CAT6a - Maximum distance of 100m at 10GB

I've known many people use CAT6 and CAT6 interchangeably when speaking. So just make sure you have the right stuff and the right distances.


Probably not too much I can help you with beyond that. We use 10GB in a several places but it's all DAC or multimode fibre between servers, switches, firewalls etc. Never done 10GB to a desktop because we've never had the need.

I would love to see 2.5Gb NIC's appearing on desktops though. Seems like the next logical step. It's starting to get more common in switches due to Wifi6 and the the pricing isn't astronomical. Better yet it works on CAT5e up to 100m!
 
You must log in or register to reply here.

Similar threads

thecomputerguy
What happens when your POE devices require more budget than is offered?
Replies
3
Views
624
NETWizz
NETWizz
thecomputerguy
NanoBeam AC Gen2 PtMP Quesitons
Replies
9
Views
2K
thecomputerguy
thecomputerguy
NETWizz
Going to be quitting Computer Networking
2
Replies
29
Views
4K
britechguy
britechguy
YeOldeStonecat
Got a difficult large network issue, wireless on new ProCurve setup
2
Replies
30
Views
5K
YeOldeStonecat
YeOldeStonecat
B
Need to get up to speed as quick as possible
2
Replies
22
Views
5K
YeOldeStonecat
YeOldeStonecat
Back
Top