9

Sonic测试研究（一）：测试架构介绍

 4 years ago
source link: https://www.sdnlab.com/24021.html
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.

作者简介：金利，盛科网络-SDN组测试工程师

Sonic项目简介

  • Sonic:Software for Open Networking in the Cloud
  • Sonic是基于Linux的开源网络操作系统，可以跑在多个不同芯片厂商交换机上
  • Sonic在2016年OCP峰会上，由微软发布
  • Sonic提供了全套的网络功能，并且都是开源的，如BGP、RDMA等协议
  • Sonic项目更多信息请查阅Github：github.com/Azure/SONic

Sonic测试框架介绍

Sonic项目专门成立了一个子项目Sonic-mgmt来负责Sonic交换机测试一块的事情

整个测试框架图如下

topo.png

设备信息

  • 这套测试框架由TestBed Server、两级Fanout交换机、Sonic被测交换机组成
  • TestBed Server上主要起两个docker（Sonic-mgmt、Docker-ptf）
    • Sonic-mgmt 主要负责调配整个测试流程，包括启动协议VM、调用ptf等
    • Docker-Ptf 主要负责收发报文以及数据平面的校验
  • Fanout交换机主要负责将Testbed Server上逻辑模拟出的端口跟被测设备物理对接起来
  • Sonic-DUT被测设备，启动Sonic软件的交换机

架构图逻辑解析

  • 物理上TestBed-Server通过两级Fanout交换机连接到被测设备
    • TestBed-Server内部通过Vlantag划分出多个逻辑端口(可以把每个逻辑端口想象成测试仪端口)
    • RootFanout进行设备级的Vlan隔离，实现多组被测设备同时进行验证
    • LeafFanout进行端口级的vlan隔离，实现Testbed逻辑端口与被测Sonic设备物理端口一一对应
  • 逻辑上TestBed-Server上通过vlan隔离出来的各个逻辑端口与物理上被测Sonic设备进行一对一端口连接

TestBed Server内部结构

TestBed_i.png

网络连接

  • TestBed Server有两个网络端口
    • Mgmt口：管理端口，连接到实验室的管理网络
    • Trunk口：数据端口，连接Fanout交换机，负责数据层面业务发包/收包验证

VM虚拟机

  • VM按照测试场景需要，由Mgmt-Sonic Docker负责调用启动，实际跑在TestBed Server上
    • VM使用的是Arista vEOS 镜像
    • VM主要负责跑各种协议，如BGP、LACP、LLDP，然后跟被测设备做协议层面的对接
    • 每个VM由10个逻辑口，8个数据口逻辑上跟交换机各个物理端口相连，另外还有一个背板口负责VM之间互联，还有一个管理口

Docker-ptf

  • Ptf(python test framework) docker由Mgmt-Sonic Docker负责调用启动，跑在TestBed Server上
    • Ptf docker可以按照实际场景，启动不同数量的逻辑端口与被测设备互联
    • Ptf docker负责测试过程中数据层面，收发报文，以及报文验证
    • Ptf 主要通过调用scapy库完成上述动作，并且可以灵活扩展，比如适配Ixia测试仪等

自动化脚本处理流程详解

  • 控制执行层面
    • 脚本以aniable-playbook的形式在Sonic-mgmt-docker上进行调用执行
    • playbook中可以包含配置下发、状态查看、配置清空等动作
    • playbook中可以通过调用ptf-docker来实现收发报文、验证报文
  • 数据层面(以server发出为例)
    • 数据层面Docker-Ptf、VM通过Server发送出来的报文都会携带不同的vlantag，用于区分内部逻辑端口（所以服务器数据出口是trunk口）
    • RootFanout交换机收到带vlantag的报文之后，通过出口vlan过滤，将携带特定vlan范围的报文发送到特定的LeafFanout交换机上(所以Root Fanout入口和出口都是trunk口，只是过滤的范围不同)
    • LeafFanout交换机收到带vlantag的报文之后，通过出口vlan过滤，并去除相应过滤tag之后，转发到被测设备的不同物理端口上，最终这个Tag对于被测设备来说是不可见的(所以LeafFanout连接RootFanout的是trunk口，连接被测设备的是Access端口)

简化逻辑图举例

Topo_1.png

  • 这个示例图是去掉fanout交换机后的逻辑topo示例，摘自官网说明文档topo中其中一个
  • PTF这里的连接端口数是可以自己定义改变的
  • VM连接数量也是可以自己定义改变的

Sonic测试脚本

PTF发包原理

  • PTF发包实际都是调用的scapy来完成，跟oftest如出一辙
  • 如果熟悉openflow oftest的小伙伴应该对这块会非常亲切

''' When receiving gratuitous ARP packet, if it was not resolved in ARP table before, SONiC should discard the request and won't add ARP entry for the GARP ''' def runTest(self): acs_mac = self.test_params['acs_mac'] pkt = simple_arp_packet(pktlen=60, eth_dst='ff:ff:ff:ff:ff:ff', eth_src='00:05:07:08:09:0a', vlan_vid=0, vlan_pcp=0, arp_op=1, ip_snd='10.10.1.7', ip_tgt='10.10.1.7', hw_snd='00:05:07:08:09:0a', hw_tgt='ff:ff:ff:ff:ff:ff', ) exp_pkt = simple_arp_packet(eth_dst='00:05:07:08:09:0a', eth_src=acs_mac, arp_op=2, ip_snd='10.10.1.2', ip_tgt='10.10.1.7', hw_tgt='00:05:07:08:09:0a', hw_snd=acs_mac, ) send_packet(self, self.test_params['port'], pkt) verify_no_packet(self, exp_pkt, self.test_params['port'])

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
 
    '''
    When receiving gratuitous ARP packet,ifit was notresolved inARP table before,
    SONiC should discard the request andwon't add ARP entry for the GARP
    '''
    def runTest(self):
        acs_mac=self.test_params['acs_mac']
        pkt=simple_arp_packet(pktlen=60,
                      eth_dst='ff:ff:ff:ff:ff:ff',
                      eth_src='00:05:07:08:09:0a',
                      vlan_vid=0,
                      vlan_pcp=0,
                      arp_op=1,
                      ip_snd='10.10.1.7',
                      ip_tgt='10.10.1.7',
                      hw_snd='00:05:07:08:09:0a',
                      hw_tgt='ff:ff:ff:ff:ff:ff',
                      )
        exp_pkt=simple_arp_packet(eth_dst='00:05:07:08:09:0a',
                        eth_src=acs_mac,
                        arp_op=2,
                        ip_snd='10.10.1.2',
                        ip_tgt='10.10.1.7',
                        hw_tgt='00:05:07:08:09:0a',
                        hw_snd=acs_mac,
                   )
        send_packet(self,self.test_params['port'], pkt)
        verify_no_packet(self,exp_pkt,self.test_params['port'])
 

Sonic-Mgmt执行脚本

  • 执行脚本、配置等操作流程实际都是调用的ansible-playbook来完成
  • 脚本都比较长，下述截取部分arpall.yml中内容

- name: wait for interfaces to be up after removed from portchannel pause: seconds=40 when: (po1 is defined) or (po2 is defined)

- name: copy test files over copy: src=roles/test/files/ptftests dest=/root delegate_to: "{{ ptf_host }}"

- name: Clear DUT arp cache command: ip -stats neigh flush all ignore_errors: yes become: yes

- name: Start PTF runner and Send correct unicast arp packets (10.10.1.3 to 10.10.1.2 with src_mac=00:06:07:08:09:00) include_tasks: ptf_runner.yml vars: ptf_test_name: ARP test ptf_test_dir: ptftests ptf_test_path: arptest.VerifyUnicastARPReply ptf_platform: remote ptf_platform_dir: ptftests ptf_test_params: - acs_mac='{{ ansible_interface_facts[intf1]['macaddress'] }}' - port='{{ intf1_indice }}' ptf_extra_options: "--relax --debug info --log-file /tmp/arptest.VerifyUnicastARPReply.{{lookup('pipe','date +%Y-%m-%d-%H:%M:%S')}}.log "

- name: Get DUT arp table switch_arptable:

- name: Check SONiC ARP table and confirm macaddress and interface are correct assert: that: - "{{ arptable['v4']['10.10.1.3']['macaddress'] == '00:06:07:08:09:00' }}" - "{{ arptable['v4']['10.10.1.3']['interface'] == intf1 }}"

- name: Clear DUT arp cache command: ip -stats neigh flush all ignore_errors: yes become: yes

# Send correct ARP request from correct interface, expecting normal behavior - name: PTF funner Send correct arp packets (10.10.1.3 to 10.10.1.2 with src_mac=00:06:07:08:09:0a) include_tasks: ptf_runner.yml vars: ptf_test_name: ARP test ptf_test_dir: ptftests ptf_test_path: arptest.ExpectReply ptf_platform: remote ptf_platform_dir: ptftests ptf_test_params: - acs_mac='{{ ansible_interface_facts[intf1]['macaddress'] }}' - port='{{ intf1_indice }}' ptf_extra_options: "--relax --debug info --log-file /tmp/arptest.VerifyARPReply.{{lookup('pipe','date +%Y-%m-%d-%H:%M:%S')}}.log "

- name: Get DUT arp table switch_arptable:

- name: Check SONiC ARP table and confirm macaddress and interface are correct assert: that: - "{{ arptable['v4']['10.10.1.3']['macaddress'] == '00:06:07:08:09:0a' }}" - "{{ arptable['v4']['10.10.1.3']['interface'] == intf1 }}"

## check DUT won't reply ARP and install ARP entry when ARP request coming from other interfaces - name: Clear DUT arp cache command: ip -stats neigh flush all ignore_errors: yes become: yes

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
 
    -name:wait forinterfaces tobe up after removed from portchannel
      pause:seconds=40
      when:(po1 isdefined)or(po2 isdefined)
 
    -name:copy test files over
      copy:src=roles/test/files/ptftests
            dest=/root
      delegate_to:"{{ptf_host}}"
 
    -name:Clear DUT arp cache
      command:ip-stats neigh flush all
      ignore_errors:yes
      become:yes
 
    -name:Start PTF runner andSend correct unicast arp packets(10.10.1.3to10.10.1.2with src_mac=00:06:07:08:09:00)
      include_tasks:ptf_runner.yml
      vars:
         ptf_test_name:ARP test
         ptf_test_dir:ptftests
         ptf_test_path:arptest.VerifyUnicastARPReply
         ptf_platform:remote
         ptf_platform_dir:ptftests
         ptf_test_params:
            -acs_mac='{{ ansible_interface_facts[intf1]['macaddress'] }}'
            -port='{{ intf1_indice }}'
         ptf_extra_options:"--relax--debug info--log-file/tmp/arptest.VerifyUnicastARPReply.{{lookup('pipe','date +%Y-%m-%d-%H:%M:%S')}}.log "
 
    -name:Get DUT arp table
      switch_arptable:
 
    -name:Check SONiC ARP table andconfirm macaddress andinterfaceare correct
      assert:
        that:
          -"{{arptable['v4']['10.10.1.3']['macaddress'] == '00:06:07:08:09:00' }}"
          -"{{arptable['v4']['10.10.1.3']['interface'] == intf1 }}"
 
    -name:Clear DUT arp cache
      command:ip-stats neigh flush all
      ignore_errors:yes
      become:yes
 
    # Send correct ARP request from correct interface, expecting normal behavior
    -name:PTF funner Send correct arp packets(10.10.1.3to10.10.1.2with src_mac=00:06:07:08:09:0a)
      include_tasks:ptf_runner.yml
      vars:
         ptf_test_name:ARP test
         ptf_test_dir:ptftests
         ptf_test_path:arptest.ExpectReply
         ptf_platform:remote
         ptf_platform_dir:ptftests
         ptf_test_params:
            -acs_mac='{{ ansible_interface_facts[intf1]['macaddress'] }}'
            -port='{{ intf1_indice }}'
         ptf_extra_options:"--relax--debug info--log-file/tmp/arptest.VerifyARPReply.{{lookup('pipe','date +%Y-%m-%d-%H:%M:%S')}}.log "
 
    -name:Get DUT arp table
      switch_arptable:
 
    -name:Check SONiC ARP table andconfirm macaddress andinterfaceare correct
      assert:
        that:
          -"{{arptable['v4']['10.10.1.3']['macaddress'] == '00:06:07:08:09:0a' }}"
          -"{{arptable['v4']['10.10.1.3']['interface'] == intf1 }}"
 
    ## check DUT won't reply ARP and install ARP entry when ARP request coming from other interfaces
    -name:Clear DUT arp cache
      command:ip-stats neigh flush all
      ignore_errors:yes
      become:yes
 

最后号召一下，有没有也在搭这套测试环境的小伙伴加我微信feng_891005，一起讨论，一个人研究太累了。


report

Recommend

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK