TCP Prague Classic ECN AQM Fallback v2

How to Read the RAG (Red-Amber-Green) Matrix

Commentary

Flows

TCP Prague Classic ECN AQM Fallback v2 - Detection RAG Matrix

Note: This Detection RAG matrix does not give an overview of the outcome of the experiments. For that, the Classic Rate Ratio (CRR) heat-map and normalized rate whisker plots should be consulted. The Detection RAG matrix is primarily an aid for algorithm tuning. The commentary below explains this point.

The key below indexes the smallest 4x5 matrices that each represent 20 experiments with different combinations of traffic flows.

Each traffic scenario is denoted by two descriptors placed either side of a colon, with the Prague flows always given first.

n: denotes n long-running flows, where n is a number, typically 0, 1, 9
L: denotes a
Low loadExponential arrival process, averaging 1 request per second for the 4Mb/s link rate, scaled pro rata for faster links
of
short web-like flowsFlow sizes Pareto distributed with α = 0.9; min 1KB, max 1MB.
1L: denotes '1' and 'L'

Example:

9:1L: denotes 9 Prague and 1 Cubic-ECN long running flow plus a low load of short web-like Cubic-ECN flows. This is the traffic scenario at the intersection of row 9: and column :1L.

Each of these 20 experiments has been run with 5x5=20 different combinations of link rate and base RTT, and in turn with different combinations of code, of traffic and of AQMs. The traffic-light colours give an intuitive feel for where parameter tuning could result in improvements.

Click 4x5 matrix To dive into detailed time-series plots tracking all the relevant metrics, click on one of the 4x5 matrices.

Click (5x5)x(4x5) matrix To view whisker plots of normalized rates and the Rate Ratio and Detection RAG matrices for one combination of AQM and host software (5x5)x(4x5)=500 experiments click on the wide margin of a matrix (highlights in red when mouse over).

		Actual AQM	AQM detected by all long Prague flows, within 10s, without significant transient errors
		Classic	L4S
		L4S	Classicor detected by a significant minority of short Prague flows
		Classic	Classic
		L4S	L4S

Code

Algorithm

Fallback v2.2 (detection only)

Fallback v2.2

RTT indep?

Traffic

Staggered?

Mixed RTTs?

↓ Bitrate (Mbps) : Base RTT (ms) →

100

One AQM

CoDel

120

200

COBALT

120

200

DualPI2

120

200

↓ Bitrate (Mbps) : Base RTT (ms) →

100

Switch AQMs

CoDel -> DualPI2

120

200

DualPI2 -> CoDel

120

200

←Cubic‑ECN

Cases where the algorithm incorrectly detects the AQM are not ideal, but they are not of concern in themselves, unless they result in reduced throughput for Classic flows (which is what the Classic Rate Ratio heat-map reveals). So far, none of the AQM detection experiments marked red or amber resulted in significantly reduced Cubic-ECN throughput. There are a number of reasons for why incorrect detection has not reduced throughput in practice:

In most cases marked red, zero long-running Cubic flows were present (*:0 or *:L flows).
Detection has been deemed to be incorrect if it has not reached the correct conclusion within about 10s. While often, given a little longer, the algorithm correctly detects the AQM. In contrast, the Classic Rate Ratio is measured once connections have stabilized. The choice of a 10s deadline was a subjective trade-off between how long it would be likely to take for incorrect detection to be noticeable to an end-user and the impact on the run-time of all the experiments. In the absence of this trade-off, the detection success tests would have been given longer to reach a conclusion. For instance, the paper that introduced TCP Cubic in 2005 considered convergence in about 100s as "a short period of time" (for a lower bandwidth-delay product than the maximum used here).
for experiments over an L4S AQM, detection was deemed to have failed if at least a significant minority of short flows switched to Classic behaviour by the time they completed. However, such transient effects are rarely detectable on plots of the resulting flow throughput, let alone detectable to an end-user.

Bob Briscoe and Asad Sajjad Ahmed