mirror of https://dev.iopsys.eu/hal/wifimngr.git synced 2025-12-20 01:11:11 +08:00

Files

Jakob Olsson 10428ea579 gitlab-ci: add tests for new functions, update docs and schemas

i.e. add/del_iface, un/subscribe, dump_beacon and monitor functions

2020-09-16 15:56:29 +02:00

27 KiB

Raw Permalink Blame History

Test Specification

Most of the functionality in wifimngr can be tested via its ubus API. Each API can be broken down into an individual test case to show full coverage is achieved.

Sections

Preqreuisites
Test Suites
Functional API Tests
Unit Tests
Functional Tests
Writing New Tests

Prerequisites

The only prerequisite for the wifimngr test suites is that libwifi has to be built for the TEST platform, a versionprepared to publish dummy data for getter API, and record setter API to a test logfile at /tmp/test.log.

Test Suites

The wifimngr build pipe has three test suites, a functional-api suite, a unit testi suite and functional test suite.

Functional API Tests

The functional API tests consists of four individual test suites, one per object Ubus under test. The functional API tests use the Ubus-API-validation command-line interface tool to invoke a method, programmatically through libubus, and validates it against the objects json-schema.

wifi

Execution ID	Method	Description	Function ID Coverage
1	status	No argument	1
2	dummy	Negative Test

wifi.ap.<name>

Execution ID	Method	Description	Function ID Coverage
1	status	No argument	2
2	stats	No argument	3
3	assoclist	No argument	4
4	stations	No argument	5
5	stations	With sta argument	5
6	list_neighbor	No argument	14
7	list_neighbor	With client argument	14
8	dump_beacon	No argument	19
9	monitor_get	No argument	9
10	monitor_get	With client argument	9
11	dummy	Negative Test

wifi.radio.<name>

Execution ID	Method	Description	Function ID Coverage
1	status	No argument	20
2	stats	No argument	21
3	scanresults	No argument	24
4	scanresults	With bssid argument	24
5	get	No argument	22
6	autochannel	No argument	25
7	dummy	Negative Test

wifi.wps

Execution ID	Method	Description	Function ID Coverage
1	status	No argument	30
2	status	With vif argument	30
3	generate_pin	No argument	31
4	validate_pin	With invalid pin argument	32
5	validate_pin	With valid pin argument	32
6	showpin	No argument	33
7	showpin	With vif argument	33
8	dummy	Negative Test

Unit Tests

The wifimngr unit tests are written in cmocka, invoking the ubus callbacks directly from the source code, which is compiled into a shared library. This means mocking the arguments of a cli or libubus invoke in a struct blob_attr *. The results of the call will be logged to the logfile at /tmp/test.log.

Execution ID	Method	Test Case Name	Function ID Coverage
1	disconnect	test_api_radio_disconnect	6
2	request_transition	test_api_req_bss_transition	16
3	request_neighbor	test_api_req_beacon_report	15
4	add_neighbor	test_api_add_nbr	12
5	start	test_api_wps_start	28
6	stop	test_api_wps_stop	29
7	add_vendor_ie	test_api_add_vendor_ie	17
8	del_vendor_ie	test_api_del_vendor_ie	18
9	del_neighbor	test_api_del_neighbor	13
11	scan	test_api_scan	23
12	setpin	test_set_wps_ap_pin	34
13	autochannel	test_api_acs	25
10	subscribe_frame	test_subscribe_frame	7
10	unsubscribe_frame	test_unsubscribe_frame	8
10	add_iface	test_add_iface	7
10	del_iface	test_del_iface	8
10	monitor_add	test_add_monitor	7
10	monitor_del	test_del_monitor	8

test_api_radio_disconnect

Description

Tests the wifimngr ubus API callback sta_disconnect(5), publishing the method disconnect.

Test Steps

Prepare the arguments as:

{ "sta": "50:31:32:33:34:35" }

Iussing a deauthenticate to a 5GHz client from the libwifi test platform.

Read the logfile and verify that the sta argument, and interface, was accurately logged.

Test Expected Results

The expected result is for the log file to have recorded a call to the disconnect function, through the interface test5 and with the argument sta as 50:31:32:33:34:35.

[2020-02-04 13:06:12] disconnect: { "ifname": "test5", "sta": "50:31:32:33:34:35" }

test_api_req_bss_transition

Description

Tests the wifimngr ubus API callback nbr_transition(5), publishing the method request_transition.

Test Steps

Prepare the arguments as:

{ "client": "50:31:32:33:34:35", "bssid": ["22:22:22:22:22:22", "11:11:11:11:11:11" ] }

Requesting a 5GHz client, 50:31:32:33:34:35, to transition from bssid 22:22:22:22:22:22 to bssid 11:11:11:11:11:11.

Read the logfile and verify that the interface and arguments were accurately logged.

Test Expected Results

The expected result is for the log file to have recorded a call to the req_bss_transition function, through the interface test5 and with the argument client as 50:31:32:33:34:35 and an array bssid with the entries "22:22:22:22:22:22","11:11:11:11:11:11".

[2020-02-04 13:06:12] req_bss_transition: { "ifname": "test5", "client": "50:31:32:33:34:35", "bssid": ["22:22:22:22:22:22","11:11:11:11:11:11",] }

test_api_req_beacon_report

Description

Tests the wifimngr ubus API callback nbr_request(5), publishing the method request_neighbor.

Test Steps

Prepare the arguments as:

{ "client": "50:31:32:33:34:35", "ssid": "Test SSID 5GHz" }

Requesting the client 50:31:32:33:34:35 with the ssid Test SSID 5GHz as a neighbor.

Read the logfile and verify that the interface and arguments were accurately logged.

Test Expected Results

The expected result is for the log file to have recorded a call to the nbr_request function, through the interface test5 and with the argument client as 50:31:32:33:34:35. Currently the ssid argument is ignored and not expected to be observed from the log file.

[2020-02-04 13:06:12] req_beacon_report: { "ifname": "test5", "client": "50:31:32:33:34:35" }

test_api_add_nbr

Description

Tests the wifimngr ubus API callback nbr_add(5), publishing the method add_neighbor.

Test Steps

Prepare the arguments as:

{ "bssid": "00:11:12:13:14:15", "channel": 36, "bssid_info", "5", "reg": 5, "phy": 5 }

Requesting the neighbor bssid 00:11:12:13:14:15, operating at channel 36 on registry 5 and phy 5 to be added to the neighbor list.

Read the logfile and verify that the interface and arguments were accurately logged.

Test Expected Results

The expected result is for the log file to have recorded a call to the add_neighbor function, through the interface test5 and with the arguments bssid as 00:11:12:13:14:15, channel as 36, phy as 5, reg as 5, bssid_info as "5".

[2020-02-04 13:06:12] add_neighbor: { "ifname": "test5", "bssid_info": 5, "reg": 5, "channel": 36, "phy": 5, "bssid": "00:11:12:13:14:15" }

test_api_wps_start

Description

Tests the wifimngr ubus API callback wps_start(5), publishing the method start to the wifi.wps object.

Test Steps

The test case has three steps, trying different argument inputs to reach as high coverage as possible.

The first part of the test case tests the role registrar, with a pin, allowing others devices to connect via pin, over the 5GHz interface.

{ "vif": "test5", "pin": "24033848", "mode", "pin", "role": "registrar" }

The second part tests signing up with the role enrollee over the 2.4GHz interface.

{ "vif": "test2", "role": "enrollee" }`

The third part of the test case tests the mode pbc, allowing other devices to synchronize over via its own push button configuration, over the 2.4GHz interface.

{ "vif": "test2", "mode": "pbc" }`

Read the logfile and verify that the interface and arguments were accurately logged.

Test Expected Results

The expected result is for the log file to have recorded a call to the start_wps function, showing the arguments accordingly, note that role registrar is represented by 0, and enrollee by 1, whereas mode pin is represented by 1 and pbc by 0.

[2020-02-04 13:06:12] start_wps: { "ifname": "test5", "pin": "24033848", "role": 4, "mode": 1 }
[2020-02-04 13:06:12] start_wps: { "ifname": "test2", "role": 1, "mode": 0 }
[2020-02-04 13:06:12] start_wps: { "ifname": "test2", "role": 0, "mode": 0 }

test_api_wps_stop

Description

Tests the wifimngr ubus API callback wps_stop(5), publishing the method stop to the wifi.wps object.

Test Steps

Prepare the arguments as:

{ "ifname": "test5" }

Stopping any active WPS activity on the 5GHz interface.

Read the logfile and verify that the interface and arguments were accurately logged.

Test Expected Results

As wps_stop(5) will call iterate all available interfaces and invoke the libwifi API wifi_stop_wps(1) for each and every one, we expect to see all (both) interfaces in the logs along with the function tag stop_wps.

[2020-02-04 13:06:12] stop_wps: { "ifname": "test2" }
[2020-02-04 13:06:12] stop_wps: { "ifname": "test5" }

test_api_add_vendor_ie

Description

Tests the wifimngr ubus API callback vsie_add(5), publishing the method add_vendor_ie to the wifi.ap.<name> object.

Test Steps

Prepare the arguments:

{ "mgmt": 5, "oui": "112233", "data": "dd03efa567" }

Appending the vendor specific information to the management frame.

Test Expected Results

The expected result is to observe the function tag add_vendor_ie in the logs, along with the prepared arguments, verifying that they have been parsed correctly by wifimngr and libwifi.

[2020-02-04 13:06:12] add_vendor_ie: { "ifname": "test5", "mgmt": 5, "oui": "112233", "data": "dd03efa567" }

test_api_del_vendor_ie

Description

Tests the wifimngr ubus API callback vsie_del(5), publishing the method del_vendor_ie to the wifi.ap.<name> object.

Test Steps

Prepare the arguments:

{ "mgmt": 5, "oui": "112233", "data": "dd03efa567" }

Deleting the vendor specific information from the management frame.

Test Expected Results

The expected result is to observe the function tag del_vendor_ie in the logs, along with the prepared arguments, verifying that they have been parsed correctly by wifimngr and libwifi.

[2020-02-04 13:06:12] del_vendor_ie: { "ifname": "test5", "mgmt": 5, "oui": "112233", "data": "dd03efa567" }

test_api_del_neighbor

Description

Tests the wifimngr ubus API callback nbr_del(5), publishing the method del_neighbor to the wifi.ap.<name> object.

Test Steps

Prepare the arguments:

{ "bssid": "50:10:00:11:22:33" }

Removing the bssid from the neighbor list.

Test Expected Results

The expected result is to observe the function tag del_neighbor in the test logs, along with the arguments, verifying that they have been parsed correctly by wifimngr and libwifi.

[2020-02-04 13:06:12] del_neighbor: { "ifname": "test5", "bssid": "50:10:00:11:22:33" }

test_api_scan

Description

Tests the wifimngr ubus API callback wl_scan(5), publishing the method scan to the wifi.radio.<name> object.

Test Steps

Prepare the arguments:

{ "ssid": "test", "bssid": "00:00:00:11:22:33", "channel": 36 }

Scanning the device using the ssid test.

Test Expected Results

The expected result is to observe the function tag scan in the test logs, along with the 5GHz interface and the argument test5 as ssid, the remaining arguments are ignored as they are currently not implemented or passed to libwifi.

[2020-02-04 13:06:12] scan: { "ifname": "test5", "ssid": "test", "bssid": "00:00:00:00:00:00", "channel": 0, "type": 0 }

test_set_wps_ap_pin

Description

Tests the wifimngr ubus API callback wps_set_ap_pin(5), publishing the method setpin to the wifi.wps object.

Test Steps

Prepare the arguments:

{ "ifname": "test5", "pin": "24033848" }

Setting the WPS pin of the 5GHz interface to 24033848.

Test Expected Results

The expected result is to observe the function tag set_wps_ap_pin in the test logs, along with the 5GHz interface and the pin.

[2020-02-04 13:06:12] set_wps_ap_pin: { "ifname": "test5", "pin": 24033848 }

test_api_acs

Description

Tests the wifimngr ubus API callback wl_autochannel(5), publishing the method autochannel to the wifi.radio.<name> object.

Test Steps

Prepare the arguments:

{ "ifname": "test5" }

Automatically selecting a channel to operate at. Currently no further arguments are passed as they are ignored programmatically.

Test Expected Results

The expected result is to observe the function tag acs in the test logs, along with the 5GHz interface.

[2020-02-04 13:06:12] acs: { "ifname": "test5" }

test_subscribe_frame

Description

Tests the wifimngr ubus API callback subscribe_frame(5), publishing the method subscribe_frame to the wifi.ap.<name> object. Successfully calling the method will result in listening to the given 802.11 frames.

Test Steps

Prepare the arguments:

{ "type": 0, "stype": 8 }

Start subscribing to 802.11 frames of stype set to WIFI_FRAME_BEACON and of type WIFI_FRAME_MGMT.

Test Expected Results

The expected result is to observe the function tag subscribe_frame in the test logs, along with the 5GHz interface and the provided type and stype arguments.

[2020-09-16 09:47:31] iface_subscribe_frame: { "ifname": "test5", "type": 0, "stype": 8 }

test_unsubscribe_frame

Description

Tests the wifimngr ubus API callback unsubscribe_frame(5), publishing the method unsubscribe_frame to the wifi.ap.<name> object. Successfully calling the method will result in no longer listening to the specified 802.11 frames.

Test Steps

Prepare the arguments:

{ "type": 0, "stype": 8 }

Stop subscribing to 802.11 frames of stype set to WIFI_FRAME_PROBE_REQ and of type WIFI_FRAME_CTRL.

Test Expected Results

The expected result is to observe the function tag unsubscribe_frame in the test logs, along with the 5GHz interface and the provided type and stype arguments.

[2020-09-16 09:47:31] iface_unsubscribe_frame: { "ifname": "test5", "type": 1, "stype": 4 }

test_add_iface

Description

Tests the wifimngr ubus API callback wl_add_iface(5), publishing the method add_iface to the wifi.radio.<name> object. Successfully calling the method will generate another wireless interface, and, if the config argument is set to true, a uci section will be generated for the interface in /etc/config/wireless.

Test Steps

Prepare the arguments:

{ "config": 1, "args": {"mode": "monitor", "name": "test" }}

Creating a uci config section and creating an interface in monitor mode, WIFI_MODE_STA with the name test.

Test Expected Results

The expected result is to observe the function tag add_iface in the test logs, along with the 5GHz interface, with wifi_mode set to 4, an argv array with device, name and mode. Additional, an uci config section attached to the test5 device for the interface test.

[2020-09-16 10:22:37] add_iface: { "ifname": "test5", "wifi_mode": "4", "argv": ["device", "test5", "mode", "monitor", "name", "test"]  }

test_del_iface

Description

Tests the wifimngr ubus API callback wl_del_iface(5), publishing the method del_iface to the wifi.radio.<name> object. Successfully calling the method will remove the wireless interface, and, if the config argument is set to true, the corresponding uci interface section in /etc/config/wireless will be deleted.

Test Steps

Prepare the arguments:

{ "config": 1, "iface": "test" }

Deleting the uci config section and the interface test.

Test Expected Results

The expected result is to observe the function tag del_iface in the test logs, along with the 5GHz interface, with the face set to test, config set to 1, and the uci section for the test interface to be removed.

[2020-09-16 10:22:37] del_iface: { "ifname": "test5", "iface": "test" }

test_monitor_add

Description

Tests the wifimngr ubus API callback sta_monitor_add(5), publishing the method monitor_add to the wifi.ap.<name> object. Successfully calling the method will add an unassociated client to be monitored for its rssi.

Test Steps

Prepare the arguments:

{ "sta": "11:22:33:44:55:66" }

Starting to monitor the unassociated wireless client with the mac address 11:22:33:44:55:66.

Test Expected Results

The expected result is to observe the function tag monitor_sta in the test logs, along with the 5GHz interface and the argument sta as 11:22:33:44:55:66, with enable set to 1.

[2020-09-16 11:20:12] monitor_sta: { "ifname": "test5", "sta": "11:22:33:44:55:66", "enable": 1 }

test_monitor_del

Description

Tests the wifimngr ubus API callback sta_monitor_del(5), publishing the method monitor_del to the wifi.ap.<name> object. Successfully calling the method will stop monitoring the specified unassociated client for its rssi.

Test Steps

Prepare the arguments:

{ "sta": "11:22:33:44:55:66" }

Stop monitoring the unassociated wireless client with the mac address 11:22:33:44:55:66.

Test Expected Results

The expected result is to observe the function tag monitor_sta in the test logs, along with the 5GHz interface and the argument sta as 11:22:33:44:55:66, with enable set to 0.

[2020-09-16 11:20:12] monitor_sta: { "ifname": "test5", "sta": "11:22:33:44:55:66", "enable": 0 }

Functional Tests

Serial ID	Test Case Name
1	test_api_events

test_api_events

Description

Tests the event registration and receive process. Wifimngr will read the file /etc/wifi.json in order to register to netlink events, in this case the netlink family easysoc and the group notify.

{
	"events" : [
		{
			"type": "wifi-event",
			"name": "cfg09ac03",
			"ifname": "test5",
			"family": "easysoc",
			"group": [
					"notify"
			]
		}
	]
}

Test Steps

As netlink events are not being published in the docker environment, the test will verify that a listener thread is created, registrates a listener and calls the libwifi API for receiving events.

As in practice the listener thread is intended to listen for events for the entire lifetime of the process, there is no break condition for the listener loop. However, in order to get a more accurate memory management report, the event loop, int wifimngr_recv_event(2) is mocked to allow break on the return code -1.

int wifimngr_recv_event(const char *ifname, void *handle)
{
	int err;

	for (;;) {
		err = wifi_recv_event((char *)ifname, handle);
		if (err < 0)
			return -1;
	}

	return 0;
}

Test Expected Results

The expected results are that wifimngr will have attempted to register a listner with the family easysoc, the group notify under the interface test5, and attempted to receive events under the same interface, read from the test log file under the function tags register_event and recv_event.

[2020-02-05 09:38:40] register_event: { "ifname": "test5", "family": "easysoc", "group": "notify" }
[2020-02-05 09:38:40] recv_event: { "ifname": "test5" }

Writing New Tests

On the addition of new APIs to wifimngr and libwifi, additional tests should be written to maintain the coverage and quality of the code.

Libwifi

On a new addition to the libwifi API, the test platform for libwifi has to be extended. The test platform for libwifi consists of two files, test.c and test.h, found under the subdirectory ./libwifi/modules/ under the easy-soc-libs repository.

The following examples assume the existence of a struct test.

struct test {
	unsigned char sta[6];
}

Getter

Assume the addition of the new API, get_client(2), taking an interface, char * as first argument, and a struct test *, filling the struct with the mac address of the first client in the interface.

The first step is to prepare some dummy data in the header file, in this case a dummy client of the type struct test. Two structs with static mac addresses are to be prepared, one for each test interface, holding static data to represent a mac address. Additionally, add a macro pointing to the address of the structs with the dummy data.

/* 5GHz dummy data */
const struct test test5_test_data = {
	.sta = {"\x66\x55\x44\x33\x22\x11"}
};

/* 5GHz "pointer" to dummy data */
#define test5_test	&test5_test_data

/* 2.4GHz dummy data */
const struct test test2_test_data = {
	.sta = {"\x11\x22\x33\x44\x55\x66"}
};

/* 2.4GHz "pointer" to dummy data */
#define test2_test	&test2_test_data

With the dummy data prepared, the function can be implemented. The test platform has a set of macros prepared to use when filling variables with dummy data, ranging from primitive types, buffers, arrays and structs. For this example the helper macro GET_TEST_BUF_TYPE(4) will be used:

#define TESTDATA(band, attr)	test ## band ## _ ## attr

...

#define GET_TEST_BUF_TYPE(o, iface, attr, type)				\
({									\
	if (ifname_to_band(iface) == 5)					\
		memcpy(o, TESTDATA(5, attr), sizeof(type));		\
	else								\
		memcpy(o, TESTDATA(2, attr), sizeof(type));		\
})

The macro will fill the first argument, o, wth the data pointed to by the concatenated variable test, 5, _ and the attr argument, forming test5_test.

Therefore, implementing our dummy getter, test_get_client(2), becomes simple.

int get_client(char *ifname, struct test *t)
{
	GET_TEST_BUF_TYPE(t, name, test, struct test);

	return 0;
}

Lastly, it should be added to the driver, assuming the driver has been updated with a get_client function pointer:

const struct wifi_driver test_driver = {
	.name = "test",
	.get_client = test_get_client,
	...
};

Setter

In the case of an addition of a new setter to the libwifi API. A log function is prepared in the test driver, logging a char * string to a test log file, /tmp/test.log, log_test(2). The first argument to log_test(2) is the char * representing the format, and the second is a va_list; all the arguments of the format. It is important the the format follows the structure

<FUNCTION_IDENTIFIER>: { <RELEVANT_ARGS> }

Assume the addition of a set_client(2) API call, the function should be implemented in the test driver, calling log_test(2) to be able to verify the struct usage and library linkage. Let's assume that the exposed Ubus API takes an argument in the form of a mac address, that would be provded as an argument, sta.

int test_set_client(char *ifname, struct test *t)
{
	log_test("set_client: { \"ifname\": \"%s\", \"sta\": \"%02x%02x%02x%02x%02x%02x\" }",
			ifname, t->sta[0], t->sta[0], t->sta[0],
			t->sta[0], t->sta[0], t->sta[0]);

	return 0;
}

This will log the entry as:

[2020-02-04 13:06:12] set_client: { "ifname": "test5", "sta": "00:11:22:33:44:55" }

Which can then be read by the corresponding test case and verified.

Writing Tests

When extending the test cases themselves, getters should be tested via the Ubus-API-validation tool for in the functional-api-test suite.

Setters without any external dependencies (i.e. other processes) should be included in the unit tests (./test/api_test_wifi_setters.c), and remaining setters in the functional-test suite (./test/function_test_wifi_event.c).

For the setter tests, a utility function has been prepared to parse the logfile for the latest occurrance of a function identifier tag, poll_test_log(2), utility function have been prepared. The function takes a FILE * as first argument, found in the struct test_env, and the function identifier as second argument, returning the json payload found in the log file, if any.

For more information on writing test cases see Continuous Integration at IOPSYS

27 KiB Raw Permalink Blame History

Test Specification

Sections

Prerequisites

Test Suites

Functional API Tests

wifi

wifi.ap.<name>

wifi.radio.<name>

wifi.wps

Unit Tests

test_api_radio_disconnect

Description

Test Steps

Test Expected Results

test_api_req_bss_transition

Description

Test Steps

Test Expected Results

test_api_req_beacon_report

Description

Test Steps

Test Expected Results

test_api_add_nbr

Description

Test Steps

Test Expected Results

test_api_wps_start

Description

Test Steps

Test Expected Results

test_api_wps_stop

Description

Test Steps

Test Expected Results

test_api_add_vendor_ie

Description

Test Steps

Test Expected Results

test_api_del_vendor_ie

Description

Test Steps

Test Expected Results

test_api_del_neighbor

Description

Test Steps

Test Expected Results

test_api_scan

Description

Test Steps

Test Expected Results

test_set_wps_ap_pin

Description

Test Steps

Test Expected Results

test_api_acs

Description

Test Steps

Test Expected Results

test_subscribe_frame

Description

Test Steps

Test Expected Results

test_unsubscribe_frame

Description

Test Steps

Test Expected Results

test_add_iface

Description

Test Steps

Test Expected Results

test_del_iface

Description

Test Steps

Test Expected Results

test_monitor_add

Description

Test Steps

Test Expected Results

test_monitor_del

27 KiB

Raw Permalink Blame History