Re: 400G forwarding - how does it work?

[Masataka Ohta <mohta () necom830 hpcl titech ac jp>]

ljwobker () gmail com wrote:

> Buffer designs are *really* hard in modern high speed chips, and
> there are always lots and lots of tradeoffs.  The "ideal" answer is
> an extremely large block of memory that ALL of the
> forwarding/queueing elements have fair/equal access to... but this
> physically looks more or less like a full mesh between the
> memory/buffering subsystem and all the forwarding engines, which
> becomes really unwieldly (expensive!) from a design standpoint.  The
> amount of memory you can practically put on the main NPU die is on
> the order of 20-200 **mega** bytes, where a single stack of HBM
> memory comes in at 4GB -- it's literally 100x the size.

I'm afraid you imply too much buffer bloat only to cause
unnecessary and unpleasant delay.

With 99% load M/M/1, 500 packets (750kB for 1500B MTU) of
buffer is enough to make packet drop probability less than
1%. With 98% load, the probability is 0.0041%.

But, there are so many router engineers who think, with
bloated buffer, packet drop probability can be zero, which
is wrong.

For example,

        https://www.broadcom.com/products/ethernet-connectivity/switching/stratadnx/bcm88690
        Jericho2 delivers a complete set of advanced features for
        the most demanding carrier, campus and cloud environments.
        The device supports low power, high bandwidth HBM packet
        memory offering up to 160X more traffic buffering compared
        with on-chip memory, enabling zero-packet-loss in heavily
        congested networks.

                                        Masataka Ohta