Add new CLI for PG drop counters part of counterpoll utility #1355

ayurkiv-nvda · 2021-01-14T00:23:06Z

Enhance counterpoll utility to support PG drop counters.
Add new CLI to enable/disable it.

Signed-off-by: Andriy Yurkiv [email protected]

depends on:
sonic-net/sonic-buildimage#6444
sonic-net/sonic-swss#1600

merge priority:
sonic-net/sonic-swss#1600
#1355
sonic-net/sonic-buildimage#6444

- What I did
Added new option for "counterpoll" utility

- How I did it
Enhance counterpoll utility for SAI_INGRESS_PRIORITY_GROUP_STAT_DROPPED_PACKETS
Add new CLI command and extend the show command.

- How to verify it
To test it use QoS test. run the pfcxontest and Pfcxofftest, drop can be triggered in either of the test.
It runs in ptf32 topo and requires RPC image.

admin@arc-switch1041:~$ counterpoll pg-drop enable  --> to enable the new counter
admin@arc-switch1041:~$ counterpoll show  --> check new INGRESS_PG_STAT_DROP counter status

Check counters

admin@arc-switch1041:~$ redis-cli -n 2
127.0.0.1:6379[2]> HGETALL COUNTERS:oid:0x1a000000000062
1) "SAI_INGRESS_PRIORITY_GROUP_STAT_XOFF_ROOM_WATERMARK_BYTES"
2) "0"
3) "SAI_INGRESS_PRIORITY_GROUP_STAT_SHARED_WATERMARK_BYTES"
4) "0"
5) "SAI_INGRESS_PRIORITY_GROUP_STAT_DROPPED_PACKETS"
6) "0"

- Previous command output (if the output of a command-line utility has changed)

admin@arc-switch1041:~$ counterpoll show
Type                        Interval (in ms)    Status
--------------------------  ------------------  --------
QUEUE_STAT                  default (10000)     enable
PORT_STAT                   default (1000)      enable
PORT_BUFFER_DROP            default (60000)     enable
RIF_STAT                    default (1000)      enable
QUEUE_WATERMARK_STAT        default (10000)     enable
PG_WATERMARK_STAT           default (10000)     enable
BUFFER_POOL_WATERMARK_STAT  default (10000)     enable

admin@arc-switch1041:~$ counterpoll
.......
Commands:
  config-db         Config DB counter commands
  port              Queue counter commands
  port-buffer-drop  Port buffer drop counter commands
  queue             Queue counter commands
  rif               RIF counter commands
  show              Show the counter configuration
  watermark         Watermark counter commands

- New command output (if the output of a command-line utility has changed)


admin@arc-switch1041:~$ counterpoll show
Type                        Interval (in ms)    Status
--------------------------  ------------------  --------
QUEUE_STAT                  default (10000)     enable
PORT_STAT                   default (1000)      enable
PORT_BUFFER_DROP            default (60000)     enable
RIF_STAT                    default (1000)      enable
QUEUE_WATERMARK_STAT        default (10000)     enable
PG_WATERMARK_STAT           default (10000)     enable
PG_DROP_STAT                default (10000)     enable         <--- new line
BUFFER_POOL_WATERMARK_STAT  default (10000)     enable

admin@arc-switch1041:~$ counterpoll
Usage: counterpoll [OPTIONS] COMMAND [ARGS]...
...............
Commands:
  config-db         Config DB counter commands
  pg-drop           Ingress PG drop counter commands     <--- new command
  port              Queue counter commands
  port-buffer-drop  Port buffer drop counter commands
  queue             Queue counter commands
  rif               RIF counter commands
  show              Show the counter configuration
  watermark         Watermark counter commands

ayurkiv-nvda · 2021-01-14T18:28:20Z

retest this please

volodymyrsamotiy · 2021-01-15T10:15:57Z

@neethajohn, could you you please review?

liat-grozovik · 2021-01-19T15:07:22Z

@neethajohn if this is needed for 201911 please add the proper label

liat-grozovik · 2021-01-20T17:29:36Z

@ayurkiv-nvda the PR description is missing some information. I have modified it a little bit but I believe the 'how to test' is should be extended with how to cause the drop counter to be incremented and then check validity.

You are referring to the show command for the counters which is great but should you verify the reset counter command as well?

Please all add content to the section of 'New command output (if the output of a command-line utility has changed)' as you do have a new command.

neethajohn · 2021-01-21T17:47:28Z

To add on to Liat's comment, I do not see the 'show'/'clear' command to view/clear the pg drop counters.

tests/counterpoll_test.py

ayurkiv-nvda · 2021-01-26T16:22:23Z

To add on to Liat's comment, I do not see the 'show'/'clear' command to view/clear the pg drop counters.

Currently 'counterpoll' utility provide few commands to work with it.
'counterpoll show' - print all counters info in one table. We already have general test for it
'counterpoll < counter type> enable'
'counterpoll < counter type> disable'
'counterpoll < counter type> interval' - three option for each counter type

There is no possibility to show only specific counter.
The same with clearing. There is no possibility to clean any counters via counterpoll cli

ayurkiv-nvda · 2021-01-26T16:40:26Z

@ayurkiv-nvda the PR description is missing some information. I have modified it a little bit but I believe the 'how to test' is should be extended with how to cause the drop counter to be incremented and then check validity.

You are referring to the show command for the counters which is great but should you verify the reset counter command as well?

Please all add content to the section of 'New command output (if the output of a command-line utility has changed)' as you do have a new command.

added

neethajohn · 2021-01-29T18:41:34Z

To add on to Liat's comment, I do not see the 'show'/'clear' command to view/clear the pg drop counters.

Currently 'counterpoll' utility provide few commands to work with it.
'counterpoll show' - print all counters info in one table. We already have general test for it
'counterpoll < counter type> enable'
'counterpoll < counter type> disable'
'counterpoll < counter type> interval' - three option for each counter type

There is no possibility to show only specific counter.
The same with clearing. There is no possibility to clean any counters via counterpoll cli

You cannot do it through 'counterpoll'. You need a separate cli to view the drop stats. Look at 'show priority-group' or 'show queue' in show/main.py. The same applies for 'clear'

ayurkiv-nvda · 2021-02-01T14:38:15Z

To add on to Liat's comment, I do not see the 'show'/'clear' command to view/clear the pg drop counters.

Currently 'counterpoll' utility provide few commands to work with it.
'counterpoll show' - print all counters info in one table. We already have general test for it
'counterpoll < counter type> enable'
'counterpoll < counter type> disable'
'counterpoll < counter type> interval' - three option for each counter type
There is no possibility to show only specific counter.
The same with clearing. There is no possibility to clean any counters via counterpoll cli

You cannot do it through 'counterpoll'. You need a separate cli to view the drop stats. Look at 'show priority-group' or 'show queue' in show/main.py. The same applies for 'clear'

Can we come up with CLI for this counters? This was not considered at the planning stage and will take a time to implement.
What about this command for show:
show priority-group dropped_packets

        Port    PG0    PG1    PG2    PG3    PG4    PG5    PG6    PG7
-----------  -----  -----  -----  -----  -----  -----  -----  -----
  Ethernet0      0      0      0      0      0      0      0      0
  Ethernet4      0      0      0      0      0      0      0      0
  Ethernet8      0      0      0      0      0      0      0      0
 ...............................................................
Ethernet116      0      0      0      0      0      0      0      0
Ethernet120      0      0      0      0      0      0      0      0
Ethernet124      0      0      0      0      0      0      0      0

And this for clear:
sonic-clear priority-group dropped_packet
?

…er in counterpoll utility Signed-off-by: Andriy Yurkiv <[email protected]>

neethajohn · 2021-02-02T04:36:18Z

To add on to Liat's comment, I do not see the 'show'/'clear' command to view/clear the pg drop counters.

Currently 'counterpoll' utility provide few commands to work with it.
'counterpoll show' - print all counters info in one table. We already have general test for it
'counterpoll < counter type> enable'
'counterpoll < counter type> disable'
'counterpoll < counter type> interval' - three option for each counter type
There is no possibility to show only specific counter.
The same with clearing. There is no possibility to clean any counters via counterpoll cli

You cannot do it through 'counterpoll'. You need a separate cli to view the drop stats. Look at 'show priority-group' or 'show queue' in show/main.py. The same applies for 'clear'

Can we come up with CLI for this counters? This was not considered at the planning stage and will take a time to implement.
What about this command for show:
show priority-group dropped_packets
        Port    PG0    PG1    PG2    PG3    PG4    PG5    PG6    PG7
-----------  -----  -----  -----  -----  -----  -----  -----  -----
  Ethernet0      0      0      0      0      0      0      0      0
  Ethernet4      0      0      0      0      0      0      0      0
  Ethernet8      0      0      0      0      0      0      0      0
 ...............................................................
Ethernet116      0      0      0      0      0      0      0      0
Ethernet120      0      0      0      0      0      0      0      0
Ethernet124      0      0      0      0      0      0      0      0
And this for clear:
sonic-clear priority-group dropped_packet
?

How about
show priority-group drop counters
sonic-clear priority-group drop counters

liat-grozovik · 2021-02-02T06:55:14Z

@neetha John<mailto:[email protected]> may I suggest to differ this req from the PR? As this was not in the initial discussion and there is no specific CLI for each type of counter we should add it as part of new PR and have this done. From: Neetha John <[email protected]> Sent: Tuesday, February 2, 2021 6:37 AM To: Azure/sonic-utilities <[email protected]> Cc: Liat Grozovik <[email protected]>; Comment <[email protected]> Subject: Re: [Azure/sonic-utilities] Add new CLI for PG drop counters part of counterpoll utility (#1355) To add on to Liat's comment, I do not see the 'show'/'clear' command to view/clear the pg drop counters. Currently 'counterpoll' utility provide few commands to work with it. 'counterpoll show' - print all counters info in one table. We already have general test for it 'counterpoll < counter type> enable' 'counterpoll < counter type> disable' 'counterpoll < counter type> interval' - three option for each counter type There is no possibility to show only specific counter. The same with clearing. There is no possibility to clean any counters via counterpoll cli You cannot do it through 'counterpoll'. You need a separate cli to view the drop stats. Look at 'show priority-group' or 'show queue' in show/main.py. The same applies for 'clear' Can we come up with CLI for this counters? This was not considered at the planning stage and will take a time to implement. What about this command for show: show priority-group dropped_packets Port PG0 PG1 PG2 PG3 PG4 PG5 PG6 PG7

…

----------- ----- ----- ----- ----- ----- ----- ----- ----- Ethernet0 0 0 0 0 0 0 0 0 Ethernet4 0 0 0 0 0 0 0 0 Ethernet8 0 0 0 0 0 0 0 0 ............................................................... Ethernet116 0 0 0 0 0 0 0 0 Ethernet120 0 0 0 0 0 0 0 0 Ethernet124 0 0 0 0 0 0 0 0 And this for clear: sonic-clear priority-group dropped_packet ? How about show priority-group drop counters sonic-clear priority-group drop counters — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#1355 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AKTABAYDM5MG65CREPXRVLDS456M7ANCNFSM4WBUB7ZQ>.

neethajohn · 2021-02-16T17:36:00Z

/AzurePipelines run

azure-pipelines · 2021-02-16T17:36:10Z

Azure Pipelines successfully started running 1 pipeline(s).

…ew FlexCounter group (#1600) Signed-off-by: Andriy Yurkiv <[email protected]> depends on : sonic-net/sonic-utilities#1355 sonic-net/sonic-buildimage#6444 What I did Added new option for "counterpoll" util Why I did it Add appropriate code to counterpoll/main.py How I verified it admin@arc-switch1041:~$ counterpoll pg-drop enable --> enable new counter admin@arc-switch1041:~$ counterpoll show --> check new PG_DROP_STAT counter status check counters admin@arc-switch1041:~$ redis-cli -n 2 127.0.0.1:6379[2]> HGETALL COUNTERS:oid:0x1a000000000062 1) "SAI_INGRESS_PRIORITY_GROUP_STAT_XOFF_ROOM_WATERMARK_BYTES" 2) "0" 3) "SAI_INGRESS_PRIORITY_GROUP_STAT_SHARED_WATERMARK_BYTES" 4) "0" 5) "SAI_INGRESS_PRIORITY_GROUP_STAT_DROPPED_PACKETS" 6) "0"

liat-grozovik · 2021-02-17T17:23:38Z

@ayurkiv-nvda can you please update submodule pointer so we can move forward with the next PR to be merged?

…ew FlexCounter group (#1600) Signed-off-by: Andriy Yurkiv <[email protected]> depends on : sonic-net/sonic-utilities#1355 sonic-net/sonic-buildimage#6444 What I did Added new option for "counterpoll" util Why I did it Add appropriate code to counterpoll/main.py How I verified it admin@arc-switch1041:~$ counterpoll pg-drop enable --> enable new counter admin@arc-switch1041:~$ counterpoll show --> check new PG_DROP_STAT counter status check counters admin@arc-switch1041:~$ redis-cli -n 2 127.0.0.1:6379[2]> HGETALL COUNTERS:oid:0x1a000000000062 1) "SAI_INGRESS_PRIORITY_GROUP_STAT_XOFF_ROOM_WATERMARK_BYTES" 2) "0" 3) "SAI_INGRESS_PRIORITY_GROUP_STAT_SHARED_WATERMARK_BYTES" 4) "0" 5) "SAI_INGRESS_PRIORITY_GROUP_STAT_DROPPED_PACKETS" 6) "0"

…er in counterpoll utility (#1355) Signed-off-by: Andriy Yurkiv <[email protected]> depends on: sonic-net/sonic-buildimage#6444 sonic-net/sonic-swss#1600 - What I did Added new option for "counterpoll" utility - How I did it Enhance counterpoll utility for SAI_INGRESS_PRIORITY_GROUP_STAT_DROPPED_PACKETS Add new CLI command and extend the show command. - How to verify it To test it use QoS test. run the pfcxontest and Pfcxofftest, drop can be triggered in either of the test. It runs in ptf32 topo and requires RPC image. admin@arc-switch1041:~$ counterpoll pg-drop enable --> to enable the new counter admin@arc-switch1041:~$ counterpoll show --> check new INGRESS_PG_STAT_DROP counter status

…er in counterpoll utility (sonic-net#1355) Signed-off-by: Andriy Yurkiv <[email protected]> depends on: sonic-net/sonic-buildimage#6444 sonic-net/sonic-swss#1600 - What I did Added new option for "counterpoll" utility - How I did it Enhance counterpoll utility for SAI_INGRESS_PRIORITY_GROUP_STAT_DROPPED_PACKETS Add new CLI command and extend the show command. - How to verify it To test it use QoS test. run the pfcxontest and Pfcxofftest, drop can be triggered in either of the test. It runs in ptf32 topo and requires RPC image. admin@arc-switch1041:~$ counterpoll pg-drop enable --> to enable the new counter admin@arc-switch1041:~$ counterpoll show --> check new INGRESS_PG_STAT_DROP counter status

…ew FlexCounter group (sonic-net#1600) Signed-off-by: Andriy Yurkiv <[email protected]> depends on : sonic-net/sonic-utilities#1355 sonic-net/sonic-buildimage#6444 What I did Added new option for "counterpoll" util Why I did it Add appropriate code to counterpoll/main.py How I verified it admin@arc-switch1041:~$ counterpoll pg-drop enable --> enable new counter admin@arc-switch1041:~$ counterpoll show --> check new PG_DROP_STAT counter status check counters admin@arc-switch1041:~$ redis-cli -n 2 127.0.0.1:6379[2]> HGETALL COUNTERS:oid:0x1a000000000062 1) "SAI_INGRESS_PRIORITY_GROUP_STAT_XOFF_ROOM_WATERMARK_BYTES" 2) "0" 3) "SAI_INGRESS_PRIORITY_GROUP_STAT_SHARED_WATERMARK_BYTES" 4) "0" 5) "SAI_INGRESS_PRIORITY_GROUP_STAT_DROPPED_PACKETS" 6) "0"

qiluo-msft · 2021-03-04T19:27:11Z

counterpoll/main.py

+@click.pass_context
+def pg_drop(ctx):
+    """  Ingress PG drop counter commands """
+    ctx.obj = swsssdk.ConfigDBConnector()


swsssdk [](start = 14, length = 7)

swsssdk is deprecated largely in sonic-utilities. Could you adapt the new ConfigDBConnector as in latest code?

is it also deprecated on 202012? this PR should go to 202012 as well thus need to know if the change should be done on specific branch only
@ayurkiv-nvda FYI

…ew FlexCounter group (sonic-net#1600) Signed-off-by: Andriy Yurkiv <[email protected]> depends on : sonic-net/sonic-utilities#1355 sonic-net/sonic-buildimage#6444 What I did Added new option for "counterpoll" util Why I did it Add appropriate code to counterpoll/main.py How I verified it admin@arc-switch1041:~$ counterpoll pg-drop enable --> enable new counter admin@arc-switch1041:~$ counterpoll show --> check new PG_DROP_STAT counter status check counters admin@arc-switch1041:~$ redis-cli -n 2 127.0.0.1:6379[2]> HGETALL COUNTERS:oid:0x1a000000000062 1) "SAI_INGRESS_PRIORITY_GROUP_STAT_XOFF_ROOM_WATERMARK_BYTES" 2) "0" 3) "SAI_INGRESS_PRIORITY_GROUP_STAT_SHARED_WATERMARK_BYTES" 4) "0" 5) "SAI_INGRESS_PRIORITY_GROUP_STAT_DROPPED_PACKETS" 6) "0"

abdosi · 2022-03-29T15:32:19Z

@ayurkiv-nvda can ypu please create PR for 201911 ? Cherry-pick has conflict

…rpoll, show/clear counters) (#2155) Signed-off-by: Andriy Yurkiv <[email protected]> Should be merged after sonic-net/sonic-swss#2263 Appropriate PR in master:#1355, #1461, #1583 What I did Added new option for "counterpoll" utility Added new CLI commands to view and clear PG dropped packet statistics. Added the new CLI commands to the command reference guide. How I did it Need to merge PG drop functionality to 201911 How to verify it admin@arc-switch1041:~$ counterpoll pg-drop enable --> to enable the new counter admin@arc-switch1041:~$ counterpoll show --> check new INGRESS_PG_STAT_DROP counter status Check counters admin@arc-switch1041:~$ redis-cli -n 2 127.0.0.1:6379[2]> HGETALL COUNTERS:oid:0x1a000000000062 1) "SAI_INGRESS_PRIORITY_GROUP_STAT_XOFF_ROOM_WATERMARK_BYTES" 2) "0" 3) "SAI_INGRESS_PRIORITY_GROUP_STAT_SHARED_WATERMARK_BYTES" 4) "0" 5) "SAI_INGRESS_PRIORITY_GROUP_STAT_DROPPED_PACKETS" 6) "0" show priority-group drop counters Ingress PG dropped packets: Port PG0 PG1 PG2 PG3 PG4 PG5 PG6 PG7 --------- ----- ----- ----- ----- ----- ----- ----- ----- Ethernet0 800 801 802 803 804 805 806 807 Ethernet4 400 401 402 403 404 405 406 407 Ethernet8 100 101 102 103 104 105 106 107 ... sonic-clear priority-group drop counters Cleared PG drop counters

ayurkiv-nvda force-pushed the pg_new_counter_cli_azure branch 2 times, most recently from b782cb2 to 68b9964 Compare January 14, 2021 15:24

ayurkiv-nvda force-pushed the pg_new_counter_cli_azure branch from 68b9964 to ecfed30 Compare January 14, 2021 18:51

This was referenced Jan 15, 2021

Enable PG drop counters by default sonic-net/sonic-buildimage#6444

Merged

Add SAI_INGRESS_PRIORITY_GROUP_STAT_DROPPED_PACKETS counter, create new FlexCounter group sonic-net/sonic-swss#1600

Merged

liat-grozovik requested a review from neethajohn January 19, 2021 15:05

liat-grozovik changed the title ~~Add new cli for SAI_INGRESS_PRIORITY_GROUP_STAT_DROPPED_PACKETS counter in counterpoll utility~~ Add new CLI for PG drop counters part of counterpoll utility Jan 20, 2021

neethajohn reviewed Jan 21, 2021

View reviewed changes

tests/counterpoll_test.py Outdated Show resolved Hide resolved

tests/counterpoll_test.py Show resolved Hide resolved

Add new cli for SAI_INGRESS_PRIORITY_GROUP_STAT_DROPPED_PACKETS count…

517cbda

…er in counterpoll utility Signed-off-by: Andriy Yurkiv <[email protected]>

ayurkiv-nvda force-pushed the pg_new_counter_cli_azure branch from ecfed30 to 517cbda Compare February 1, 2021 15:47

neethajohn approved these changes Feb 10, 2021

View reviewed changes

liat-grozovik approved these changes Feb 10, 2021

View reviewed changes

neethajohn merged commit 9332b8e into sonic-net:master Feb 17, 2021

liat-grozovik added the Request for 202012 Branch label Feb 17, 2021

neethajohn added the Request for 201911 Branch label Feb 17, 2021

vaibhavhd mentioned this pull request Feb 23, 2021

Advance sonic-utilities submodule to include recent changes sonic-net/sonic-buildimage#6843

Closed

4 tasks

yxieca added the Included in 202012 Branch label Feb 23, 2021

qiluo-msft reviewed Mar 4, 2021

View reviewed changes

ayurkiv-nvda mentioned this pull request May 10, 2022

[201911] [Flex Counters] add CLI for PG drop packets counters (counterpoll, show/clear counters) #2155

Merged

ayurkiv-nvda deleted the pg_new_counter_cli_azure branch May 30, 2023 21:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add new CLI for PG drop counters part of counterpoll utility #1355

Add new CLI for PG drop counters part of counterpoll utility #1355

ayurkiv-nvda commented Jan 14, 2021 •

edited

Loading

ayurkiv-nvda commented Jan 14, 2021

volodymyrsamotiy commented Jan 15, 2021

liat-grozovik commented Jan 19, 2021

liat-grozovik commented Jan 20, 2021

neethajohn commented Jan 21, 2021

ayurkiv-nvda commented Jan 26, 2021 •

edited

Loading

ayurkiv-nvda commented Jan 26, 2021

neethajohn commented Jan 29, 2021

ayurkiv-nvda commented Feb 1, 2021

neethajohn commented Feb 2, 2021

liat-grozovik commented Feb 2, 2021 via email

neethajohn commented Feb 16, 2021

azure-pipelines bot commented Feb 16, 2021

liat-grozovik commented Feb 17, 2021

qiluo-msft Mar 4, 2021 •

edited

Loading

liat-grozovik Mar 4, 2021

abdosi commented Mar 29, 2022

Add new CLI for PG drop counters part of counterpoll utility #1355

Add new CLI for PG drop counters part of counterpoll utility #1355

Conversation

ayurkiv-nvda commented Jan 14, 2021 • edited Loading

ayurkiv-nvda commented Jan 14, 2021

volodymyrsamotiy commented Jan 15, 2021

liat-grozovik commented Jan 19, 2021

liat-grozovik commented Jan 20, 2021

neethajohn commented Jan 21, 2021

ayurkiv-nvda commented Jan 26, 2021 • edited Loading

ayurkiv-nvda commented Jan 26, 2021

neethajohn commented Jan 29, 2021

ayurkiv-nvda commented Feb 1, 2021

neethajohn commented Feb 2, 2021

liat-grozovik commented Feb 2, 2021 via email

neethajohn commented Feb 16, 2021

azure-pipelines bot commented Feb 16, 2021

liat-grozovik commented Feb 17, 2021

qiluo-msft Mar 4, 2021 • edited Loading

Choose a reason for hiding this comment

liat-grozovik Mar 4, 2021

Choose a reason for hiding this comment

abdosi commented Mar 29, 2022

ayurkiv-nvda commented Jan 14, 2021 •

edited

Loading

ayurkiv-nvda commented Jan 26, 2021 •

edited

Loading

qiluo-msft Mar 4, 2021 •

edited

Loading