Skip to content

Testing

Jump to bottom
Stefan Schneider edited this page Jan 29, 2020 · 9 revisions

Weekly pilot testing

This page describes the steps of the weekly manual testing performed by UPB every Wednesday at 9:30am (just before the weekly call). If you are interested to join us, please talk to Stefan or Manuel.

Pointers

Environments

Walkthrough

1. Setup

Terminal to monitor SMP-CC

watch curl -X GET fgcn-tango-smp-ctrl.cs.upb.de/api/v1/ssmstatus

Logging

  • Open Graylog (use Safari)
  • Queries:
source:int-sp-ath
source:int-sp-ath AND container_name:tng-ssm-*
source:int-sp-ath AND container_name:tng-fsm-*

SONATA SP Dashboard

2. Package and On-board

# package latest
cd tng-industry-pilot/sdk-projects
./pack.sh
# enable tng-cli
export SP_PATH=http://int-sp-ath.5gtango.eu
# list existing packages
tng-cli package -l
# delete old packages
tng-cli package -r <uuid>
# on-board packages
tng-cli package -u eu.5gtango.tng-smpilot-ns1-k8s.0.1.tgo 
tng-cli package -u eu.5gtango.tng-smp-ns2-k8s-mdc-eids.0.9.tgo
# list service uuids
tng-cli service --descriptor

3. Prepare and upload the policy (if not already there)

Important: After deleting, packaging, and onboarding NS2 again, also a new policy has to be uploaded. Even if there is already a policy that points to the exact same NS2 (same vendor, name, version).´

  • Delete any existing policy under https://int-sp-ath.5gtango.eu/service-platform/policies/runtime-policies
    • If it can't be deleted, check any NS2 service instance (may be terminated) and switch the "Runtime policy associated"-toggle to off
    • Then proceed to upload the policy again
    • If deletion doesn't work, simply increment the policies version number to avoid having 2x the same policy (not allowed)
  • Policy file: tng-industry-pilot/policies/industrial-pilot-Security-Policy.json
  • Update policy file to match latest NS2 which was just on-boarded (vendor.name.version)
  • Upload with:
# upload
tng-cli policy -c policies/industrial-pilot-Security-Policy.json
# check if policy is uploaded
tng-cli policy -g <uuid_retruned_after_uplaod>
# should return a yaml version of the policy
  • Check: Portal -> SP -> Policies -> Runtime Policies
  • Set policy as Default by hitting the red circle next to the policy in the portal

4. Instantiate

  1. Instantiate NS1
# production NS1
tng-cli service -n sm-test-ns1 -i <ns1_service_uuid>
# qurantine NS1
tng-cli service -n sm-test-ns1-q -i <ns1_service_uuid>

  1. Get IPs etc. from NS1 to be used as parameters in NS2 Portal -> Network Services -> Instances -> (Select sm-test-ns1) -> (Select smpilot-cc) -> Copy Floating IP

  2. Update instantiation parameter sdk-projects/inst_params_ns2.yml

MQTT_BROKER_HOST: <ns1_cc_floating_ip>  # update
QUARANTINE_MQTT_BROKER_HOST: <quarantine_ns1_cc_floating_ip>  # update
SMB_USER_NAME: Alice  # don't change
SMB_CLIENT_NAME: IMMS  # don't change
  1. Instantiate NS2
tng-cli service -n sm-test-ns2 --params_file inst_params_ns2.yml -i <ns2_service_uuid>

5. Enable IMMS an check that traffic arrives in EAE

Do the following on a machine connected to the VPN. You need to get the IP of the MDC CNF from the portal (sm-test-ns2 -> msf-vnf1 -> FloatingIP).

# pull IMMS image
docker pull sonatanfv/vnf-dt:latest
# check NS2 MSF floating IP on portal
docker run -d --rm -p 15001:15001 --name vnf-dt -e DT_EM63_SHARE_HOST=<ns2-mdc-ip> sonatanfv/vnf-dt:latest

Check if traffic arrives:

  • Get EAE IP (sm-test-ns1 -> eae-vnf -> FloatingIP)
  • Browse to: http://<eae_ip>:3000/
  • Login: admin/tango
  • Top left -> Home Dropdown -> Manufacturing Edge Analytics
  • Dashboard should show up with traffic arriving (i.e., Parts counter increasing)

Check if traffic arrives in quarantine service:

  • Get EAE IP (sm-test-ns1-q -> eae-vnf -> FloatingIP)
  • Browse to: http://<eae_ip>:3000/
  • Login: admin/tango
  • Top left -> Home Dropdown -> Manufacturing Edge Analytics
  • Dashboard should show up with NO traffic arriving

6. Manually trigger intrusion

From any machine connected to the VPN:

curl -X PUT fgcn-tango-smp-ctrl.cs.upb.de/api/v1/ssmstatus -d uuid=<put_the_service_instance_uuid_here> -d quarantaine=1
  • Check Graylog using source:int-sp-ath AND source:int-sp-ath AND container_name:tng-fsm-*
  • Find something like:
INFO:fsm-mdc:Response to request generated:{'envs': [{'cdu_id': 'cdu01-76b084df-86b4-4eb1-87d5-fb72d219b7b7', 'envs': {'MQTT_BROKER_HOST': '10.200.16.41', 'QUARANTINE_MQTT_BROKER_HOST': '10.200.16.41', 'NORMAL_MQTT_BROKER_HOST': '10.200.16.39'}}], 'status': 'COMPLETED', 'error': 'None'}
  • Check the two Grafana dashboard. Traffic should now arrive in quarantine dashboard.

7. Reset intrusion

curl -X PUT fgcn-tango-smp-ctrl.cs.upb.de/api/v1/ssmstatus -d uuid=<put_the_service_instance_uuid_here> -d quarantaine=0
  • Check again the logs and Grafana.

8. Automatic trigger of intrusion

  1. Open Prometheus to see if policy becomes active (http://int-sp-ath.5gtango.eu:9090/graph)
    • Click Alerts there should be an list entry like lhc_vnf2_cdu02_ip0_greater0_7bfa2e2c which was dynamically created by the policy. (green = not triggered yet)
  2. Trigger the intrusion (from any machine connected to the VPN):
smbclient -L <floating_ip_of_mfs_vnf1> -U eve

Optional: To make it more dramatic (for video), run nmap first:

nmap -v -A -Pn <floating_ip_of_mfs_vnf1>
  1. There should be a metric ip0 in Prometheus which represents the intrusion trigger.
  2. Verify: Traffic should again arrive in the quarantine Grafana (see section about manually triggered intrusion).

Issues found

https://github.com/sonata-nfv/tng-industrial-pilot/issues/330

Clone this wiki locally