-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
The feature "polling only configured ports buffer queue" will break SNMP #17448
Comments
telemetry has same issue? |
Updated in original text. |
@qiluo-msft i see you provided step by step for repro but is there a sonic-mgmt test that can run to reproduce and confirm it will no longer be an issue not just for 202205? |
@qiluo-msft to review the PR. |
7 tasks
qiluo-msft
pushed a commit
to sonic-net/sonic-snmpagent
that referenced
this issue
Jan 9, 2024
…b_buffers is set to true (#303) This happened because the MIB assumed that half the queues configured are for mcast. When create_only_config_db_buffers is set to true this is not the case **- What I did** The ciscoSwitchQosMIB MIB assumed that all the counter are configured and that half of the configured queues are for mcast. This is no longer true, the feature "polling only configured ports buffer queue" make it possible for port to not have MC counters. This wrong assumption caused issue sonic-net/sonic-buildimage#17448 To fix this, I instead used the BUFFER_MAX_PARAM_TABLE to find the max possible queues **- How I did it** **- How to verify it** Inside the SNMP docker run snmp walk: $ snmpwalk -v2c -c msft 10.64.247.240 1.3.6.1.4.1.9.9.580.1.5.5.1.4.1 Check that the results is not missing counters from any queue
mssonicbld
pushed a commit
to mssonicbld/sonic-snmpagent
that referenced
this issue
Jan 9, 2024
…b_buffers is set to true (sonic-net#303) This happened because the MIB assumed that half the queues configured are for mcast. When create_only_config_db_buffers is set to true this is not the case **- What I did** The ciscoSwitchQosMIB MIB assumed that all the counter are configured and that half of the configured queues are for mcast. This is no longer true, the feature "polling only configured ports buffer queue" make it possible for port to not have MC counters. This wrong assumption caused issue sonic-net/sonic-buildimage#17448 To fix this, I instead used the BUFFER_MAX_PARAM_TABLE to find the max possible queues **- How I did it** **- How to verify it** Inside the SNMP docker run snmp walk: $ snmpwalk -v2c -c msft 10.64.247.240 1.3.6.1.4.1.9.9.580.1.5.5.1.4.1 Check that the results is not missing counters from any queue
mssonicbld
pushed a commit
to sonic-net/sonic-snmpagent
that referenced
this issue
Jan 9, 2024
…b_buffers is set to true (#303) This happened because the MIB assumed that half the queues configured are for mcast. When create_only_config_db_buffers is set to true this is not the case **- What I did** The ciscoSwitchQosMIB MIB assumed that all the counter are configured and that half of the configured queues are for mcast. This is no longer true, the feature "polling only configured ports buffer queue" make it possible for port to not have MC counters. This wrong assumption caused issue sonic-net/sonic-buildimage#17448 To fix this, I instead used the BUFFER_MAX_PARAM_TABLE to find the max possible queues **- How I did it** **- How to verify it** Inside the SNMP docker run snmp walk: $ snmpwalk -v2c -c msft 10.64.247.240 1.3.6.1.4.1.9.9.580.1.5.5.1.4.1 Check that the results is not missing counters from any queue
mssonicbld
pushed a commit
to mssonicbld/sonic-snmpagent
that referenced
this issue
Jan 9, 2024
…b_buffers is set to true (sonic-net#303) This happened because the MIB assumed that half the queues configured are for mcast. When create_only_config_db_buffers is set to true this is not the case **- What I did** The ciscoSwitchQosMIB MIB assumed that all the counter are configured and that half of the configured queues are for mcast. This is no longer true, the feature "polling only configured ports buffer queue" make it possible for port to not have MC counters. This wrong assumption caused issue sonic-net/sonic-buildimage#17448 To fix this, I instead used the BUFFER_MAX_PARAM_TABLE to find the max possible queues **- How I did it** **- How to verify it** Inside the SNMP docker run snmp walk: $ snmpwalk -v2c -c msft 10.64.247.240 1.3.6.1.4.1.9.9.580.1.5.5.1.4.1 Check that the results is not missing counters from any queue
qiluo-msft
pushed a commit
to sonic-net/sonic-snmpagent
that referenced
this issue
Jan 9, 2024
…b_buffers is set to true (#303) (#305) This happened because the MIB assumed that half the queues configured are for mcast. When create_only_config_db_buffers is set to true this is not the case **- What I did** The ciscoSwitchQosMIB MIB assumed that all the counter are configured and that half of the configured queues are for mcast. This is no longer true, the feature "polling only configured ports buffer queue" make it possible for port to not have MC counters. This wrong assumption caused issue sonic-net/sonic-buildimage#17448 To fix this, I instead used the BUFFER_MAX_PARAM_TABLE to find the max possible queues **- How I did it** **- How to verify it** Inside the SNMP docker run snmp walk: $ snmpwalk -v2c -c msft 10.64.247.240 1.3.6.1.4.1.9.9.580.1.5.5.1.4.1 Check that the results is not missing counters from any queue Co-authored-by: DavidZagury <[email protected]>
qiluo-msft
pushed a commit
to sonic-net/sonic-mgmt
that referenced
this issue
Jan 10, 2024
## Approach #### What is the motivation for this PR? ADO : 26336010 Modify test to detect issue sonic-net/sonic-buildimage#17448. #### How did you do it? Currently test_snmp_queue only verifies that each interface does have queue counters associated with it. It does not verify if all the expected queue counters are present. Modify test_snmp_queue to add 2 additional checks: 1. Ensure that queue index with queue configuration in config_db, is present for each interface. 2. Ensure that the number of Unicast queue counters for each interface in 'show queue counters <interface name> is the same as the number of queue indexes in the SNMP result. #### How did you verify/test it? Test case passes for single-asic/multi-asic/Chassis platforms. Test case passed with: 202205/202305/202311/master images Test case fails when run on device with image which has the issue: sonic-net/sonic-buildimage#17448. ``` snmp_q_idx = int(queue_idx) + 1 if str(snmp_q_idx) not in v['queues'][direction_type]: pytest.fail("\ Expected queue index %d not present in SNMP \ > result for interface %s" % (snmp_q_idx, v['name'])) E Failed: Expected queue index 6 not present in SNMP result for interface Ethernet13/3 .. snmp/test_snmp_queue.py:87: Failed -------------------------------------------------------------------------------------------------- generated xml file: /data/sonic-mgmt-int/tests/logs/tr.xml -------------------------------------------------------------------------------------------------- =================================================================================================================== short test summary info ==================================================================================================================== FAILED snmp/test_snmp_queue.py::test_snmp_queues[str3-7060-acs-1] - Failed: Expected queue index 6 not present in SNMP result for interface Ethernet13/3 ================================================================================================================== 1 failed in 248.30 seconds ================================================================================================================== INFO:root:Can not get Allure report URL. Please check logs ```
mssonicbld
pushed a commit
to mssonicbld/sonic-mgmt
that referenced
this issue
Jan 17, 2024
…t#11180) ## Approach #### What is the motivation for this PR? ADO : 26336010 Modify test to detect issue sonic-net/sonic-buildimage#17448. #### How did you do it? Currently test_snmp_queue only verifies that each interface does have queue counters associated with it. It does not verify if all the expected queue counters are present. Modify test_snmp_queue to add 2 additional checks: 1. Ensure that queue index with queue configuration in config_db, is present for each interface. 2. Ensure that the number of Unicast queue counters for each interface in 'show queue counters <interface name> is the same as the number of queue indexes in the SNMP result. #### How did you verify/test it? Test case passes for single-asic/multi-asic/Chassis platforms. Test case passed with: 202205/202305/202311/master images Test case fails when run on device with image which has the issue: sonic-net/sonic-buildimage#17448. ``` snmp_q_idx = int(queue_idx) + 1 if str(snmp_q_idx) not in v['queues'][direction_type]: pytest.fail("\ Expected queue index %d not present in SNMP \ > result for interface %s" % (snmp_q_idx, v['name'])) E Failed: Expected queue index 6 not present in SNMP result for interface Ethernet13/3 .. snmp/test_snmp_queue.py:87: Failed -------------------------------------------------------------------------------------------------- generated xml file: /data/sonic-mgmt-int/tests/logs/tr.xml -------------------------------------------------------------------------------------------------- =================================================================================================================== short test summary info ==================================================================================================================== FAILED snmp/test_snmp_queue.py::test_snmp_queues[str3-7060-acs-1] - Failed: Expected queue index 6 not present in SNMP result for interface Ethernet13/3 ================================================================================================================== 1 failed in 248.30 seconds ================================================================================================================== INFO:root:Can not get Allure report URL. Please check logs ```
7 tasks
mssonicbld
pushed a commit
to mssonicbld/sonic-mgmt
that referenced
this issue
Jan 17, 2024
…t#11180) ## Approach #### What is the motivation for this PR? ADO : 26336010 Modify test to detect issue sonic-net/sonic-buildimage#17448. #### How did you do it? Currently test_snmp_queue only verifies that each interface does have queue counters associated with it. It does not verify if all the expected queue counters are present. Modify test_snmp_queue to add 2 additional checks: 1. Ensure that queue index with queue configuration in config_db, is present for each interface. 2. Ensure that the number of Unicast queue counters for each interface in 'show queue counters <interface name> is the same as the number of queue indexes in the SNMP result. #### How did you verify/test it? Test case passes for single-asic/multi-asic/Chassis platforms. Test case passed with: 202205/202305/202311/master images Test case fails when run on device with image which has the issue: sonic-net/sonic-buildimage#17448. ``` snmp_q_idx = int(queue_idx) + 1 if str(snmp_q_idx) not in v['queues'][direction_type]: pytest.fail("\ Expected queue index %d not present in SNMP \ > result for interface %s" % (snmp_q_idx, v['name'])) E Failed: Expected queue index 6 not present in SNMP result for interface Ethernet13/3 .. snmp/test_snmp_queue.py:87: Failed -------------------------------------------------------------------------------------------------- generated xml file: /data/sonic-mgmt-int/tests/logs/tr.xml -------------------------------------------------------------------------------------------------- =================================================================================================================== short test summary info ==================================================================================================================== FAILED snmp/test_snmp_queue.py::test_snmp_queues[str3-7060-acs-1] - Failed: Expected queue index 6 not present in SNMP result for interface Ethernet13/3 ================================================================================================================== 1 failed in 248.30 seconds ================================================================================================================== INFO:root:Can not get Allure report URL. Please check logs ```
mssonicbld
pushed a commit
to mssonicbld/sonic-mgmt
that referenced
this issue
Jan 17, 2024
…t#11180) ## Approach #### What is the motivation for this PR? ADO : 26336010 Modify test to detect issue sonic-net/sonic-buildimage#17448. #### How did you do it? Currently test_snmp_queue only verifies that each interface does have queue counters associated with it. It does not verify if all the expected queue counters are present. Modify test_snmp_queue to add 2 additional checks: 1. Ensure that queue index with queue configuration in config_db, is present for each interface. 2. Ensure that the number of Unicast queue counters for each interface in 'show queue counters <interface name> is the same as the number of queue indexes in the SNMP result. #### How did you verify/test it? Test case passes for single-asic/multi-asic/Chassis platforms. Test case passed with: 202205/202305/202311/master images Test case fails when run on device with image which has the issue: sonic-net/sonic-buildimage#17448. ``` snmp_q_idx = int(queue_idx) + 1 if str(snmp_q_idx) not in v['queues'][direction_type]: pytest.fail("\ Expected queue index %d not present in SNMP \ > result for interface %s" % (snmp_q_idx, v['name'])) E Failed: Expected queue index 6 not present in SNMP result for interface Ethernet13/3 .. snmp/test_snmp_queue.py:87: Failed -------------------------------------------------------------------------------------------------- generated xml file: /data/sonic-mgmt-int/tests/logs/tr.xml -------------------------------------------------------------------------------------------------- =================================================================================================================== short test summary info ==================================================================================================================== FAILED snmp/test_snmp_queue.py::test_snmp_queues[str3-7060-acs-1] - Failed: Expected queue index 6 not present in SNMP result for interface Ethernet13/3 ================================================================================================================== 1 failed in 248.30 seconds ================================================================================================================== INFO:root:Can not get Allure report URL. Please check logs ```
mssonicbld
pushed a commit
to sonic-net/sonic-mgmt
that referenced
this issue
Jan 17, 2024
## Approach #### What is the motivation for this PR? ADO : 26336010 Modify test to detect issue sonic-net/sonic-buildimage#17448. #### How did you do it? Currently test_snmp_queue only verifies that each interface does have queue counters associated with it. It does not verify if all the expected queue counters are present. Modify test_snmp_queue to add 2 additional checks: 1. Ensure that queue index with queue configuration in config_db, is present for each interface. 2. Ensure that the number of Unicast queue counters for each interface in 'show queue counters <interface name> is the same as the number of queue indexes in the SNMP result. #### How did you verify/test it? Test case passes for single-asic/multi-asic/Chassis platforms. Test case passed with: 202205/202305/202311/master images Test case fails when run on device with image which has the issue: sonic-net/sonic-buildimage#17448. ``` snmp_q_idx = int(queue_idx) + 1 if str(snmp_q_idx) not in v['queues'][direction_type]: pytest.fail("\ Expected queue index %d not present in SNMP \ > result for interface %s" % (snmp_q_idx, v['name'])) E Failed: Expected queue index 6 not present in SNMP result for interface Ethernet13/3 .. snmp/test_snmp_queue.py:87: Failed -------------------------------------------------------------------------------------------------- generated xml file: /data/sonic-mgmt-int/tests/logs/tr.xml -------------------------------------------------------------------------------------------------- =================================================================================================================== short test summary info ==================================================================================================================== FAILED snmp/test_snmp_queue.py::test_snmp_queues[str3-7060-acs-1] - Failed: Expected queue index 6 not present in SNMP result for interface Ethernet13/3 ================================================================================================================== 1 failed in 248.30 seconds ================================================================================================================== INFO:root:Can not get Allure report URL. Please check logs ```
mssonicbld
pushed a commit
to sonic-net/sonic-mgmt
that referenced
this issue
Jan 17, 2024
## Approach #### What is the motivation for this PR? ADO : 26336010 Modify test to detect issue sonic-net/sonic-buildimage#17448. #### How did you do it? Currently test_snmp_queue only verifies that each interface does have queue counters associated with it. It does not verify if all the expected queue counters are present. Modify test_snmp_queue to add 2 additional checks: 1. Ensure that queue index with queue configuration in config_db, is present for each interface. 2. Ensure that the number of Unicast queue counters for each interface in 'show queue counters <interface name> is the same as the number of queue indexes in the SNMP result. #### How did you verify/test it? Test case passes for single-asic/multi-asic/Chassis platforms. Test case passed with: 202205/202305/202311/master images Test case fails when run on device with image which has the issue: sonic-net/sonic-buildimage#17448. ``` snmp_q_idx = int(queue_idx) + 1 if str(snmp_q_idx) not in v['queues'][direction_type]: pytest.fail("\ Expected queue index %d not present in SNMP \ > result for interface %s" % (snmp_q_idx, v['name'])) E Failed: Expected queue index 6 not present in SNMP result for interface Ethernet13/3 .. snmp/test_snmp_queue.py:87: Failed -------------------------------------------------------------------------------------------------- generated xml file: /data/sonic-mgmt-int/tests/logs/tr.xml -------------------------------------------------------------------------------------------------- =================================================================================================================== short test summary info ==================================================================================================================== FAILED snmp/test_snmp_queue.py::test_snmp_queues[str3-7060-acs-1] - Failed: Expected queue index 6 not present in SNMP result for interface Ethernet13/3 ================================================================================================================== 1 failed in 248.30 seconds ================================================================================================================== INFO:root:Can not get Allure report URL. Please check logs ```
mssonicbld
pushed a commit
to mssonicbld/sonic-snmpagent
that referenced
this issue
Jan 20, 2024
…b_buffers is set to true (sonic-net#303) This happened because the MIB assumed that half the queues configured are for mcast. When create_only_config_db_buffers is set to true this is not the case **- What I did** The ciscoSwitchQosMIB MIB assumed that all the counter are configured and that half of the configured queues are for mcast. This is no longer true, the feature "polling only configured ports buffer queue" make it possible for port to not have MC counters. This wrong assumption caused issue sonic-net/sonic-buildimage#17448 To fix this, I instead used the BUFFER_MAX_PARAM_TABLE to find the max possible queues **- How I did it** **- How to verify it** Inside the SNMP docker run snmp walk: $ snmpwalk -v2c -c msft 10.64.247.240 1.3.6.1.4.1.9.9.580.1.5.5.1.4.1 Check that the results is not missing counters from any queue
StormLiangMS
pushed a commit
to sonic-net/sonic-snmpagent
that referenced
this issue
Jan 20, 2024
…b_buffers is set to true (#303) (#309) This happened because the MIB assumed that half the queues configured are for mcast. When create_only_config_db_buffers is set to true this is not the case **- What I did** The ciscoSwitchQosMIB MIB assumed that all the counter are configured and that half of the configured queues are for mcast. This is no longer true, the feature "polling only configured ports buffer queue" make it possible for port to not have MC counters. This wrong assumption caused issue sonic-net/sonic-buildimage#17448 To fix this, I instead used the BUFFER_MAX_PARAM_TABLE to find the max possible queues **- How I did it** **- How to verify it** Inside the SNMP docker run snmp walk: $ snmpwalk -v2c -c msft 10.64.247.240 1.3.6.1.4.1.9.9.580.1.5.5.1.4.1 Check that the results is not missing counters from any queue Co-authored-by: DavidZagury <[email protected]>
mssonicbld
pushed a commit
to sonic-net/sonic-mgmt
that referenced
this issue
Jan 22, 2024
## Approach #### What is the motivation for this PR? ADO : 26336010 Modify test to detect issue sonic-net/sonic-buildimage#17448. #### How did you do it? Currently test_snmp_queue only verifies that each interface does have queue counters associated with it. It does not verify if all the expected queue counters are present. Modify test_snmp_queue to add 2 additional checks: 1. Ensure that queue index with queue configuration in config_db, is present for each interface. 2. Ensure that the number of Unicast queue counters for each interface in 'show queue counters <interface name> is the same as the number of queue indexes in the SNMP result. #### How did you verify/test it? Test case passes for single-asic/multi-asic/Chassis platforms. Test case passed with: 202205/202305/202311/master images Test case fails when run on device with image which has the issue: sonic-net/sonic-buildimage#17448. ``` snmp_q_idx = int(queue_idx) + 1 if str(snmp_q_idx) not in v['queues'][direction_type]: pytest.fail("\ Expected queue index %d not present in SNMP \ > result for interface %s" % (snmp_q_idx, v['name'])) E Failed: Expected queue index 6 not present in SNMP result for interface Ethernet13/3 .. snmp/test_snmp_queue.py:87: Failed -------------------------------------------------------------------------------------------------- generated xml file: /data/sonic-mgmt-int/tests/logs/tr.xml -------------------------------------------------------------------------------------------------- =================================================================================================================== short test summary info ==================================================================================================================== FAILED snmp/test_snmp_queue.py::test_snmp_queues[str3-7060-acs-1] - Failed: Expected queue index 6 not present in SNMP result for interface Ethernet13/3 ================================================================================================================== 1 failed in 248.30 seconds ================================================================================================================== INFO:root:Can not get Allure report URL. Please check logs ```
mssonicbld
pushed a commit
to mssonicbld/sonic-snmpagent
that referenced
this issue
Jan 23, 2024
…b_buffers is set to true (sonic-net#303) This happened because the MIB assumed that half the queues configured are for mcast. When create_only_config_db_buffers is set to true this is not the case **- What I did** The ciscoSwitchQosMIB MIB assumed that all the counter are configured and that half of the configured queues are for mcast. This is no longer true, the feature "polling only configured ports buffer queue" make it possible for port to not have MC counters. This wrong assumption caused issue sonic-net/sonic-buildimage#17448 To fix this, I instead used the BUFFER_MAX_PARAM_TABLE to find the max possible queues **- How I did it** **- How to verify it** Inside the SNMP docker run snmp walk: $ snmpwalk -v2c -c msft 10.64.247.240 1.3.6.1.4.1.9.9.580.1.5.5.1.4.1 Check that the results is not missing counters from any queue
mssonicbld
pushed a commit
to sonic-net/sonic-snmpagent
that referenced
this issue
Jan 23, 2024
…b_buffers is set to true (#303) This happened because the MIB assumed that half the queues configured are for mcast. When create_only_config_db_buffers is set to true this is not the case **- What I did** The ciscoSwitchQosMIB MIB assumed that all the counter are configured and that half of the configured queues are for mcast. This is no longer true, the feature "polling only configured ports buffer queue" make it possible for port to not have MC counters. This wrong assumption caused issue sonic-net/sonic-buildimage#17448 To fix this, I instead used the BUFFER_MAX_PARAM_TABLE to find the max possible queues **- How I did it** **- How to verify it** Inside the SNMP docker run snmp walk: $ snmpwalk -v2c -c msft 10.64.247.240 1.3.6.1.4.1.9.9.580.1.5.5.1.4.1 Check that the results is not missing counters from any queue
This was referenced Mar 6, 2024
qiluo-msft
pushed a commit
to sonic-net/sonic-mgmt
that referenced
this issue
Apr 16, 2024
## Description of PR This test is checking that buffer queue counters inside snmp docker are inline when using create_only_config_db_counters optimization. Summary: Accompanies "Fix SNMP dropping some of the queue counter when create_only_config_db_buffers is set to true" (sonic-net/sonic-snmpagent#303) which fixes the issue: "The feature "polling only configured ports buffer queue" will break SNMP" (sonic-net/sonic-buildimage#17448). ### Type of change - [ ] Bug fix - [ ] Testbed and Framework(new/improvement) - [x] Test case(new/improvement) ## Approach #### What is the motivation for this PR? To enhance the bug fix mentioned above, solving an issue with buffer queue counters optimization. #### How did you do it? - Set "create_only_config_db_buffers" to true in config db, to create only relevant counters - Remove one of the buffer queues, Ethernet0|3-4 is chosen arbitrary - Using snmpwalk compare number of queue counters on Ethernet0, assuming there will be 8 less after removing the buffer. (Assuming unicast only, 4 counters for each queue in this case) #### How did you verify/test it? Run the test multiple times on various setups. Run the test while reverting the fix mentioned above to see the different result. ### Supported testbed topology if it's a new test case? Any
8 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Description
The issue is found after the feature sonic-net/sonic-swss#2143 is available on 202205 branch. Since the original PR is reverted, and reworked and merged. Please make sure the feature is in the image and enabled by configuration if needed.
Steps to reproduce the issue:
Describe the results you received:
Describe the results you expected:
Output of
show version
:Output of
show techsupport
:ignored
The text was updated successfully, but these errors were encountered: