Skip to content
Snippets Groups Projects
testspec.md 11 KiB
Newer Older
  • Learn to ignore specific revisions
  • Yalu Zhang's avatar
    Yalu Zhang committed
    # Test Specification
    
    Most of the functionality in *dslmngr* can be tested via its UBUS API. Each
    API can be broken down into an individual test case to show full coverage is
    achieved.
    
    ## Prerequisites
    
    The only prerequisite for the *dslmngr* test suites is that *libdsl* has to be
    built for the TEST platform, returning dummy data for getter APIs.
    
    ## Test Suites
    
    The *dslmngr* build pipe has three test suites, a functional-api suite, a unit
    test suite and a functional test suite.
    
    ### Functional API Tests
    
    The functional API test suite consists of eight test cases verifying output of
    status and statistics returned by UBUS calls. *dslmngr* and ubusd are started
    before the whole test cases' invoke. Each test case issues a UBUS call and saves
    the output to a temporary file. Then the contents of the file are analyzed by
    using JSON-C library APIs.
    
    #### dsl.line.<number>
    
    | Execution ID	| Method      	| Description 	| Function ID Coverage			|
    | :---			| :---	   		| :---			| :---							|
    | 1				| status     	| No argument  	| [1](./api/dsl.line.md#status) |
    | 2				| stats     	| No argument	  | [2](./api/dsl.line.md#stats)  |
    | 3				| stats     	| Interval	    | [3](./api/dsl.line.md#stats_interval) |
    
    | 9                             | configure     | DSL_config          | [9](./api/dsl.line.md#configure) |
    
    Yalu Zhang's avatar
    Yalu Zhang committed
    
    #### dsl.channel.<number>
    
    | Execution ID	| Method      	| Description 	| Function ID Coverage			|
    | :---			| :---	   		| :---			| :---							|
    | 4				| status     	| No argument  	| [4](./api/dsl.channel.md#status) |
    | 5				| stats     	| No argument	  | [5](./api/dsl.channel.md#stats)  |
    | 6				| stats     	| Interval	    | [6](./api/dsl.channel.md#stats_interval) |
    
    #### dsl
    
    | Execution ID	| Method      	| Description 	| Function ID Coverage			|
    | :---			| :---	   		| :---			| :---							|
    | 7				| status     	| No argument  	| [7](./api/dsl.md#status) |
    | 8				| stats     	| No argument	  | [8](./api/dsl.md#stats)  |
    
    
    #### fast.line.<number>
    
    | Execution ID  | Method        | Description   | Function ID Coverage                  |
    | :---                  | :---                  | :---                  | :---                                                  |
    | 10                             | status        | No argument   | [10](./api/fast.line.md#status) |
    | 11                             | stats         | No argument     | [11](./api/fast.line.md#stats)  |
    | 12                             | stats         | Interval          | [12](./api/fast.line.md#stats_interval) |
    
    #### atm.link.<number>
    
    | Execution ID  | Method        | Description   | Function ID Coverage                  |
    | :---                  | :---                  | :---                  | :---                                                  |
    | 13                             | status        | No argument   | [13](./api/atm.link.md#status) |
    | 14                             | stats         | No argument     | [14](./api/atm.link.md#stats)  |
    | 15                            | configure     | ATM_config          | [15](./api/atm.link.md#configure) |
    
    #### ptm.link.<number>
    
    | Execution ID  | Method        | Description   | Function ID Coverage                  |
    | :---                  | :---                  | :---                  | :---                                                  |
    | 16                             | status        | No argument   | [16](./api/ptm.link.md#status) |
    
    
    Yalu Zhang's avatar
    Yalu Zhang committed
    ### Unit Tests
    
    The *dslmngr* unit tests are written in CMocka, invoking the UBUS callbacks directly from the source code, which is compiled
    into a shared library. This means mocking the arguments of a CLI or libubus invoke in a `struct blob_attr *`.
    
    The output of the UBUS callbacks are retrieved by manipulation of the function ubus_send_reply(). Then the output is analyzed
    and verified by each test case.
    
    
    | Execution ID	| Method      	   | Test Case Name 	                   | Function ID Coverage			                  |
    
    Yalu Zhang's avatar
    Yalu Zhang committed
    | :---    | :---                   | :---                                | :---                                          |
    
    | 1				| line_status     	 | test_api_dsl_line_status            | [1](./api/dsl.line.md#status)            |
    | 2				| line_stats     	 | test_api_dsl_line_stats             | [2](./api/dsl.line.md#stats)             |
    
    Yalu Zhang's avatar
    Yalu Zhang committed
    | 3				| line_stats_interval  	 | test_api_dsl_line_stats_interval    | [3](./api/dsl.line.md#stats_interval)    |
    | 4				| channel_status         | test_api_dsl_channel_status         | [4](./api/dsl.channel.md#status)         |
    
    | 5				| channel_stats     	 | test_api_dsl_channel_stats          | [5](./api/dsl.channel.md#stats)          |
    
    Yalu Zhang's avatar
    Yalu Zhang committed
    | 6				| channel_stats_interval | test_api_dsl_channel_stats_interval | [6](./api/dsl.channel.md#stats_interval) |
    | 7				| dsl_status             | test_api_dsl_status                 | [7](./api/dsl.md#status)                 |
    | 8				| dsl_stats              | test_api_dsl_stats                  | [8](./api/dsl.md#stats)                  |
    
    | 9                             | dsl_configure          | test_api_dsl_configure              | [9](./api/dsl.line.md#configure)         |
    | 10                            | dsl_configure_fail     | test_api_dsl_configure_fail         | [9](./api/dsl.line.md#configure)         |
    | 11                            | atm_configure          | test_api_atm_configure              | [15](./api/atm.link.md#configure)        |
    | 12                            | atm_configure_fail     | test_api_atm_configure_fai          | [15](./api/atm.link.md#configure)        |
    
    Yalu Zhang's avatar
    Yalu Zhang committed
    
    #### test_api_dsl_line_status
    
    ##### Description
    
    Test the *dslmngr* UBUS API callback `dsl_line_status()`, providing the method [line status](./api/dsl.line.md#status).
    
    ##### Test Steps
    
    - Call `dsl_line_status()`
    - Analyze and verify the output
    
    ##### Expected Results
    
    Parameters' values included in the returned JSON object of the output shall be equal to those returned by the corresponding
    stub API in *libdsl* building for TEST platform.
    
    #### test_api_dsl_line_stats
    
    ##### Description
    
    Test the *dslmngr* UBUS API callback `dsl_line_stats()`, providing the method [line stats](./api/dsl.line.md#stats).
    
    ##### Test Steps
    
    - Call `dsl_line_stats()`
    - Analyze and verify the output
    
    ##### Expected Results
    
    Parameters' values included in the returned JSON object of the output shall be equal to those returned by the corresponding
    stub API in *libdsl* building for TEST platform.
    
    #### test_api_dsl_line_stats_interval
    
    ##### Description
    
    Test the *dslmngr* UBUS API callback `dsl_line_stats()`, providing the method [line stats](./api/dsl.line.md#stats).
    
    ##### Test Steps
    
    - Call `dsl_line_stats()` with intervals `total`, `showtime`, `lastshowtime`, `currentday`, and `quarterhour` sequentially
    - Analyze and verify the output
    
    ##### Expected Results
    
    Parameters' values included in the returned JSON object of the output shall be equal to those returned by the corresponding
    stub API in *libdsl* building for TEST platform.
    
    #### test_api_dsl_channel_status
    
    ##### Description
    
    Test the *dslmngr* UBUS API callback `dsl_channel_status()`, providing the method [channel status](./api/dsl.channel.md#status).
    
    ##### Test Steps
    
    - Call `dsl_channel_status()`
    - Analyze and verify the output
    
    ##### Expected Results
    
    Parameters' values included in the returned JSON object of the output shall be equal to those returned by the corresponding
    stub API in *libdsl* building for TEST platform.
    
    #### test_api_dsl_channel_stats
    
    ##### Description
    
    Test the *dslmngr* UBUS API callback `dsl_channel_stats()`, providing the method [channel stats](./api/dsl.channel.md#stats).
    
    ##### Test Steps
    
    - Call `dsl_channel_stats()`
    - Analyze and verify the output
    
    ##### Expected Results
    
    Parameters' values included in the returned JSON object of the output shall be equal to those returned by the corresponding
    stub API in *libdsl* building for TEST platform.
    
    #### test_api_dsl_channel_stats_interval
    
    ##### Description
    
    Test the *dslmngr* UBUS API callback `dsl_channel_stats()`, providing the method [channel stats](./api/dsl.channel.md#stats).
    
    ##### Test Steps
    
    - Call `dsl_channel_stats()` with intervals `total`, `showtime`, `lastshowtime`, `currentday`, and `quarterhour` sequentially
    - Analyze and verify the output
    
    ##### Expected Results
    
    Parameters' values included in the returned JSON object of the output shall be equal to those returned by the corresponding
    stub API in *libdsl* building for TEST platform.
    
    #### test_api_dsl_status
    
    ##### Description
    
    Test the *dslmngr* UBUS API callback `dsl_status_all()`, providing the method [dsl status](./api/dsl.md#status).
    
    ##### Test Steps
    
    - Call `dsl_status_all()`
    - Analyze and verify the output
    
    ##### Expected Results
    
    Parameters' values included in the returned JSON object of the output shall be equal to those returned by the corresponding
    stub API in *libdsl* building for TEST platform.
    
    #### test_api_dsl_stats
    
    ##### Description
    
    Test the *dslmngr* UBUS API callback `dsl_stats_all()`, providing the method [dsl stats](./api/dsl.md#stats).
    
    ##### Test Steps
    
    - Call `dsl_stats_all()`
    - Analyze and verify the output
    
    ##### Expected Results
    
    Parameters' values included in the returned JSON object of the output shall be equal to those returned by the corresponding
    stub API in *libdsl* building for TEST platform.
    
    
    #### dsl_configure
    
    ##### Description
    
    Test the *dslmngr* UBUS API callback `dsl_line_configure()`, providing the method [dsl configure](./api/dsl.line.md#configure).
    
    ##### Test Steps
    
    - Prepare valid input parameters
    - Call `dsl_line_configure()`
    - Analyze and verify the output
    
    ##### Expected Results
    
    The dumped input parameters by the corresponding libdsl API must be as same as those prepared in th first step. And the
    returned value must indicate success.
    
    #### dsl_configure_fail
    
    ##### Description
    
    Test the *dslmngr* UBUS API callback `dsl_line_configure()`, providing the method [dsl configure](./api/dsl.line.md#configure).
    
    ##### Test Steps
    
    - Prepare invalid input parameters
    - Call `dsl_line_configure()`
    - Analyze and verify the output
    
    ##### Expected Results
    
    The dumped input parameters by the corresponding libdsl API must be as same as those prepared in th first step. And the
    returned value must indicate failure.
    
    #### atm_configure
    
    ##### Description
    
    Test the *dslmngr* UBUS API callback `atm_link_configure()`, providing the method [atm configure](./api/atm.link.md#configure).
    
    ##### Test Steps
    
    - Prepare valid input parameters
    - Call `atm_link_configure()`
    - Analyze and verify the output
    
    ##### Expected Results
    
    The dumped input parameters by the corresponding libdsl API must be as same as those prepared in th first step. And the
    returned value must indicate success.
    
    #### atm_configure_fail
    
    ##### Description
    
    Test the *dslmngr* UBUS API callback `atm_link_configure()`, providing the method [atm configure](./api/atm.link.md#configure).
    
    ##### Test Steps
    
    - Prepare valid input parameters
    - Call `atm_link_configure()`
    - Analyze and verify the output
    
    ##### Expected Results
    
    The dumped input parameters by the corresponding libdsl API must be as same as those prepared in th first step. And the
    returned value must indicate failure.